Sep 17 10:51:21.067699 ip-10-0-129-205 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 17 10:51:21.067841 ip-10-0-129-205 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 17 10:51:21.067940 ip-10-0-129-205 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 17 10:51:21.068481 ip-10-0-129-205 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 17 10:51:31.166527 ip-10-0-129-205 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 17 10:51:31.166549 ip-10-0-129-205 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c8c987e30fa14fefa4c18e441b1023e8 --
Sep 17 10:53:24.575207 ip-10-0-129-205 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 10:53:24.946619 ip-10-0-129-205 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 10:53:24.946619 ip-10-0-129-205 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 10:53:24.946619 ip-10-0-129-205 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 10:53:24.946619 ip-10-0-129-205 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 10:53:24.946619 ip-10-0-129-205 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 10:53:24.947295 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.946704    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 10:53:24.951248 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951225    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 10:53:24.951248 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951249    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 10:53:24.951248 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951254    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 10:53:24.951248 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951257    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951260    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951263    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951267    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951270    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951272    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951275    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951278    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951281    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951283    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951286    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951289    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951291    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951294    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951296    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951303    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951305    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951308    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951310    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951313    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 10:53:24.951371 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951315    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951319    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951321    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951330    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951334    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951337    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951340    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951342    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951345    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951348    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951351    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951353    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951356    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951358    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951362    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951365    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951368    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951371    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951373    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 10:53:24.951850 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951376    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951379    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951381    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951384    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951386    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951389    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951391    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951396    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951400    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951403    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951406    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951408    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951411    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951414    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951416    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951419    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951421    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951425    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951429    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 10:53:24.952333 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951432    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951435    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951438    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951441    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951444    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951447    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951450    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951452    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951455    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951457    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951460    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951463    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951465    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951468    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951470    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951473    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951476    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951480    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951482    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 10:53:24.952797 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951485    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951489    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951492    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951495    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951497    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951500    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951502    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951505    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951507    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951510    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951512    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951514    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951517    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951520    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951522    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951525    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951528    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951530    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.951533    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952218    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 10:53:24.953343 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952226    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952229    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952232    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952235    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952238    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952241    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952243    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952246    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952249    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952252    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952254    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952257    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952259    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952262    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952264    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952267    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952272    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952276    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952279    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952281    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 10:53:24.953828 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952285    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952288    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952291    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952294    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952297    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952300    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952303    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952305    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952308    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952310    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952313    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952316    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952319    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952321    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952324    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952326    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952329    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952331    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952334    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952337    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 10:53:24.954432 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952339    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952342    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952344    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952347    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952349    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952352    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952354    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952357    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952359    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952362    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952364    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952367    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952369    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952372    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952374    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952377    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952379    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952382    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952385    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 10:53:24.954934 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952388    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952391    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952394    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952396    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952399    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952401    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952404    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952406    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952409    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952411    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952414    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952417    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952419    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952421    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952424    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952426    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952429    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952431    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952435    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952437    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 10:53:24.955405 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952440    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952442    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952444    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952447    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952451    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952454    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952457    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952460    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952463    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952466    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952468    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952471    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952473    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952478    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952481    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952483    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952486    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952488    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952491    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 10:53:24.955889 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952870    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952875    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952879    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952882    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952885    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952888    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952892    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952913    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952916    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952918    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952921    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952923    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952926    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952928    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952931    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952933    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952936    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952938    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952943    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952946    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 10:53:24.956364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952948    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952951    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952954    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952956    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952959    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952961    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952965    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952967    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952970    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952972    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952975    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952978    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952981    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952985    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952988    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952991    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952994    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952997    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.952999    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 10:53:24.956846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953002    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953004    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953008    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953011    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953014    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953017    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953019    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953022    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953024    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953027    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953029    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953032    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953035    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953038    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953040    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953043    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953045    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953048    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953050    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953053    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 10:53:24.957315 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953056    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953058    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953061    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953063    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953066    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953069    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953071    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953074    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953077    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953079    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953082    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953085    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953087    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953089    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953092    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953094    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953096    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953099    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953101    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953104    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 10:53:24.957848 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953106    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953109    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953111    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953115    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953118    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953120    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953122    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953125    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953127    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953129    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953132    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953136    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953138    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953141    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953143    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953145    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953148    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953150    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953153    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953155    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 10:53:24.958350 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953217    2564 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953252    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953258    2564 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953262    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953266    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953269    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953273    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953277    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953279    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953282    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953286    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953289    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953291    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953294    2564 flags.go:64] FLAG: --cgroup-root=""
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953297    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953303    2564 flags.go:64] FLAG: --client-ca-file=""
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953307    2564 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953311    2564 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953318    2564 flags.go:64] FLAG: --cluster-domain=""
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953321    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953324    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953327    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953331    2564 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953335    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 10:53:24.958832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953338    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953341    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953344    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953347    2564 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953350    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953353    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953356    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953359    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953363    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953366    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953369    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953372    2564 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953374    2564 flags.go:64] FLAG: --enable-server="true"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953378    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953383    2564 flags.go:64] FLAG: --event-burst="100"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953386    2564 flags.go:64] FLAG: --event-qps="50"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953389    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953392    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953394    2564 flags.go:64] FLAG: --eviction-hard=""
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953398    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953401    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953403    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953406    2564 flags.go:64] FLAG: --eviction-soft=""
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953410    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953413    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 10:53:24.959435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953416    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953418    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953421    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953424    2564 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953427    2564 flags.go:64] FLAG: --feature-gates=""
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953431    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953435    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953438    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953440    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953443    2564 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953446    2564 flags.go:64] FLAG: --help="false"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953449    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-129-205.ec2.internal"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953452    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953455    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953457    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953461    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953464    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953467    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953470    2564 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953472    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953475    2564 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953478    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953481    2564 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953484    2564 flags.go:64] FLAG: --kube-reserved=""
Sep 17 10:53:24.960108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953486    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953489    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953492    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953494    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953497    2564 flags.go:64] FLAG: --lock-file=""
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953500    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953504    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953506    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953512    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953514    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953517    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953519    2564 flags.go:64] FLAG: --logging-format="text"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953522    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953525    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953529    2564 flags.go:64] FLAG: --manifest-url=""
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953532    2564 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953536    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953538    2564 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953542    2564 flags.go:64] FLAG: --max-pods="110"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953545    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953548    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953550    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953553    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953556    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953559    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 10:53:24.960977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953562    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953568    2564 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953571    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953574    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953576    2564 flags.go:64] FLAG: --pod-cidr=""
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953580    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953585    2564 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953588    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953591    2564 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953594    2564 flags.go:64] FLAG: --port="10250"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953596    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953599    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-029a269a29b41a9ed"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953602    2564 flags.go:64] FLAG: --qos-reserved=""
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953606    2564 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953609    2564 flags.go:64] FLAG: --register-node="true"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953612    2564 flags.go:64] FLAG: --register-with-taints=""
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953615    2564 flags.go:64] FLAG: --registry-burst="10"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953618    2564 flags.go:64] FLAG: --registry-qps="5"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953621    2564 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953624    2564 flags.go:64] FLAG: --reserved-memory=""
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953632    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953635    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953640    2564 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953643    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953646    2564 flags.go:64] FLAG: --runonce="false"
Sep 17 10:53:24.961858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953649    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953652    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953655    2564 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953658    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953660    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953663    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953666    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953669    2564 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953672    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953675    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953678    2564 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953680    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953683    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953687    2564 flags.go:64] FLAG: --system-cgroups=""
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953690    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953695    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953697    2564 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953700    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953704    2564 flags.go:64] FLAG: --tls-min-version=""
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953707    2564 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953711    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953713    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953716    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953719    2564 flags.go:64] FLAG: --v="2"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953722    2564 flags.go:64] FLAG: --version="false"
Sep 17 10:53:24.962538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953726    2564 flags.go:64] FLAG: --vmodule=""
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953729    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.953732    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953841    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953846    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953849    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953852    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953854    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953857    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953859    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953864    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953868    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953871    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953874    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953877    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953880    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953883    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953886    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953889    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 10:53:24.963163 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953892    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953908    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953912    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953914    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953917    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953920    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953922    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953925    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953929    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953932    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953934    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953937    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953940    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953943    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953946    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953948    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953951    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953954    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953959    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 10:53:24.963642 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953962    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953965    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953968    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953970    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953973    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953976    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953979    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953981    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953984    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953987    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953989    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953993    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953996    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.953999    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954002    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954004    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954007    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954010    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954013    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954015    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 10:53:24.964194 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954018    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954020    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954023    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954025    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954027    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954030    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954033    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954036    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954038    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954041    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954045    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954048    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954052    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954056    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954060    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954063    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954065    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954068    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954070    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954073    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 10:53:24.964667 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954076    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954078    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954081    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954083    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954086    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954089    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954091    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954094    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954096    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954099    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954102    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954105    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954107    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954110    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954113    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954115    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954118    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954120    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954123    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954125    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 10:53:24.965191 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954128    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 10:53:24.965734 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954130    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 10:53:24.965734 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954135    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 10:53:24.965734 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.954138    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 10:53:24.965734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.954634    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 10:53:24.965734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.962739    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 10:53:24.965734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.962755    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 10:53:24.965734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.962768    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 10:53:24.965734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.962774    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 10:53:24.965734 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962827    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 10:53:24.965734 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962832    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 10:53:24.965734 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962835    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 10:53:24.965734 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962838    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 10:53:24.965734 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962841    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 10:53:24.965734 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962844    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962847    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962850    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962853    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962856    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962858    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962861    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962864    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962866    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962869    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962873    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962877    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962880    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962883    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962886    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962889    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962891    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962912    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962916    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 10:53:24.966095 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962919    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962921    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962924    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962927    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962930    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962934    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962937    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962939    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962942    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962945    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962947    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962950    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962952    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962955    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962958    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962961    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962963    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962966    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962969    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962971    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 10:53:24.966561 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962974    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962976    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962979    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962981    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962985    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962989    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962991    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962994    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962996    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.962999    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963001    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963004    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963006    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963009    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963011    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963014    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963017    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963020    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963023    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963026    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 10:53:24.967069 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963028    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963031    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963033    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963036    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963038    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963041    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963043    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963046    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963048    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963051    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963054    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963056    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963059    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963061    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963065    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963069    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963071    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963074    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963077    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963080    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 10:53:24.967560 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963083    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 10:53:24.968066 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963085    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 10:53:24.968066 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963087    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 10:53:24.968066 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963090    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 10:53:24.968066 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963093    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 10:53:24.968066 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963095    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 10:53:24.968066 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963097    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 10:53:24.968066 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963100    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 10:53:24.968066 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963102    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 10:53:24.968066 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963106    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 10:53:24.968066 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963108    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 10:53:24.968066 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963111    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 10:53:24.968066 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963113    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 10:53:24.968066 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963116    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 10:53:24.968066 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963118    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 10:53:24.968066 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.963123    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963228    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963234    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963239    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963242    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963245    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963248    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963251    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963254    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963257    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963260    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963262    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963265    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963268    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963270    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963273    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963275    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963278    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963281    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963283    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 10:53:24.968446 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963286    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963288    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963291    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963294    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963296    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963299    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963302    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963305    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963308    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963310    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963313    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963315    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963318    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963321    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963323    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963334    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963337    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963340    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963343    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 10:53:24.968945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963347    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963350    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963352    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963355    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963358    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963361    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963363    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963367    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963369    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963372    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963374    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963377    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963379    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963382    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963384    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963387    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963390    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963392    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963395    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963398    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 10:53:24.969417 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963400    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963403    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963405    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963408    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963410    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963413    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963416    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963418    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963421    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963423    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963426    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963428    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963431    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963433    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963436    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963438    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963441    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963443    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963445    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963448    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 10:53:24.969910 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963451    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963453    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963456    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963458    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963460    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963463    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963465    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963468    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963470    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963472    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963475    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963477    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963480    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963484    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963488    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963490    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963493    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963495    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963498    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963500    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 10:53:24.970396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:24.963503    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 10:53:24.970874 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.963507    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 10:53:24.970874 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.963624    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 10:53:24.970874 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.966020    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 10:53:24.970874 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.966956    2564 server.go:1019] "Starting client certificate rotation"
Sep 17 10:53:24.970874 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.967081    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 10:53:24.970874 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.967746    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 10:53:24.970874 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.969310    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 10:53:24.973624 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.973608    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 10:53:24.987224 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.987129    2564 log.go:25] "Validated CRI v1 runtime API"
Sep 17 10:53:24.991183 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.991168    2564 log.go:25] "Validated CRI v1 image API"
Sep 17 10:53:24.992072 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.992054    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 10:53:24.997344 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.997327    2564 fs.go:135] Filesystem UUIDs: map[625ceb1c-4f82-4d8f-b21d-1fbcd53cf07e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e07466c2-27e9-4af6-a2f2-a71831bb119d:/dev/nvme0n1p3]
Sep 17 10:53:24.997414 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:24.997359    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 10:53:25.002680 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.002656    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 10:53:25.005039 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.004935    2564 manager.go:216] Machine: {Timestamp:2026-09-17 10:53:25.003641549 +0000 UTC m=+0.320004694 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101624 MemoryCapacity:32812146688 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec223df4147362058e1fe99651db1e64 SystemUUID:ec223df4-1473-6205-8e1f-e99651db1e64 BootID:c8c987e3-0fa1-4fef-a4c1-8e441b1023e8 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406073344 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ed:e9:22:e3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ed:e9:22:e3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:2e:71:e5:7f:cc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812146688 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 10:53:25.005039 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.005034    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 10:53:25.005158 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.005140    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 10:53:25.008202 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.008178    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 10:53:25.008354 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.008205    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-205.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 10:53:25.008857 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.008847    2564 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 10:53:25.008889 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.008860    2564 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 10:53:25.008889 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.008868    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 10:53:25.009389 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.009379    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 10:53:25.010113 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.010103    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 10:53:25.013673 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.013662    2564 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 10:53:25.013784 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.013775    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 10:53:25.013824 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.013803    2564 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 10:53:25.013824 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.013812    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 10:53:25.013824 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.013822    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 10:53:25.013940 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.013833    2564 kubelet.go:398] "Adding apiserver pod source"
Sep 17 10:53:25.013940 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.013841    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 10:53:25.014207 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.014194    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 10:53:25.014255 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.014217    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 10:53:25.016230 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.016205    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 17 10:53:25.018149 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.018136    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 10:53:25.018196 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.018166    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 10:53:25.019473 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.019459    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 10:53:25.019518 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.019482    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 10:53:25.019518 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.019489    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 10:53:25.019518 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.019495    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 10:53:25.019518 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.019504    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 10:53:25.019518 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.019510    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 10:53:25.019647 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.019531    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 10:53:25.019647 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.019536    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 10:53:25.019647 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.019543    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 10:53:25.019647 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.019548    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 10:53:25.019647 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.019560    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 10:53:25.019647 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.019565    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 10:53:25.020282 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.020270    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 10:53:25.020317 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.020283    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 10:53:25.024031 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.024018    2564 server.go:1268] "Started kubelet"
Sep 17 10:53:25.024253 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.024165    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 10:53:25.024253 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.024173    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 10:53:25.024384 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.024284    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 10:53:25.024545 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.024518    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 17 10:53:25.024625 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.024580    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-205.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 17 10:53:25.024753 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.024731    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-205.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 17 10:53:25.025020 ip-10-0-129-205 systemd[1]: Started Kubernetes Kubelet.
Sep 17 10:53:25.027985 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.027270    2564 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 10:53:25.028769 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.028753    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 10:53:25.030213 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.030191    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 10:53:25.032565 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.032551    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 10:53:25.032637 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.032568    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 10:53:25.033011 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.032740    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-205.ec2.internal\" not found"
Sep 17 10:53:25.033194 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.033178    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 10:53:25.033194 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.033195    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 10:53:25.033316 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.033287    2564 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 10:53:25.033316 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.033298    2564 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 10:53:25.033316 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.033289    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 10:53:25.033771 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.033752    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 10:53:25.033771 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.033771    2564 factory.go:55] Registering systemd factory
Sep 17 10:53:25.033870 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.033780    2564 factory.go:223] Registration of the systemd container factory successfully
Sep 17 10:53:25.034069 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.034054    2564 factory.go:153] Registering CRI-O factory
Sep 17 10:53:25.034102 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.034074    2564 factory.go:223] Registration of the crio container factory successfully
Sep 17 10:53:25.034874 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.034329    2564 factory.go:103] Registering Raw factory
Sep 17 10:53:25.034874 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.034362    2564 manager.go:1196] Started watching for new ooms in manager
Sep 17 10:53:25.035526 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.035503    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-205.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 17 10:53:25.035601 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.034683    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-205.ec2.internal.18d61649126e2316  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-205.ec2.internal,UID:ip-10-0-129-205.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-205.ec2.internal,},FirstTimestamp:2026-09-17 10:53:25.023982358 +0000 UTC m=+0.340345511,LastTimestamp:2026-09-17 10:53:25.023982358 +0000 UTC m=+0.340345511,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-205.ec2.internal,}"
Sep 17 10:53:25.035751 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.035737    2564 manager.go:349] Starting recovery of all containers
Sep 17 10:53:25.038284 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.038260    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 17 10:53:25.047705 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.047415    2564 manager.go:354] Recovery completed
Sep 17 10:53:25.052318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.052292    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9q29m"
Sep 17 10:53:25.052386 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.052339    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 10:53:25.054056 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.054038    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 10:53:25.054109 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.054073    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 10:53:25.054109 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.054089    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasSufficientPID"
Sep 17 10:53:25.054629 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.054617    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 10:53:25.054671 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.054631    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 10:53:25.054671 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.054653    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 10:53:25.054734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.054649    2564 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 10:53:25.055388 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.055373    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 10:53:25.055437 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.055401    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 10:53:25.055437 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.055413    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasSufficientPID"
Sep 17 10:53:25.055973 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.055893    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-205.ec2.internal.18d6164914390df8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-205.ec2.internal,UID:ip-10-0-129-205.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-205.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-205.ec2.internal,},FirstTimestamp:2026-09-17 10:53:25.054057976 +0000 UTC m=+0.370421204,LastTimestamp:2026-09-17 10:53:25.054057976 +0000 UTC m=+0.370421204,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-205.ec2.internal,}"
Sep 17 10:53:25.056850 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.056833    2564 policy_none.go:49] "None policy: Start"
Sep 17 10:53:25.056850 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.056852    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 10:53:25.056966 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.056864    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 10:53:25.058844 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.058834    2564 policy_none.go:47] "Start"
Sep 17 10:53:25.060343 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.060322    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9q29m"
Sep 17 10:53:25.095481 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.095463    2564 manager.go:337] "Starting Device Plugin manager"
Sep 17 10:53:25.122245 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.095499    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 10:53:25.122245 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.095509    2564 server.go:87] "Starting device plugin registration server"
Sep 17 10:53:25.122245 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.095784    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 10:53:25.122245 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.095796    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 10:53:25.122245 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.095914    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 10:53:25.122245 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.095991    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 10:53:25.122245 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.096000    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 10:53:25.122245 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.096493    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 10:53:25.122245 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.096542    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-205.ec2.internal\" not found"
Sep 17 10:53:25.134855 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.134824    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 10:53:25.136174 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.136161    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 10:53:25.136241 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.136210    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 10:53:25.136241 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.136235    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 10:53:25.136346 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.136278    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 17 10:53:25.138920 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.138877    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 10:53:25.196324 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.196303    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 10:53:25.198364 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.198306    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 10:53:25.198364 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.198342    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 10:53:25.198364 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.198353    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasSufficientPID"
Sep 17 10:53:25.198531 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.198376    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.206772 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.206756    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.206843 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.206777    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-205.ec2.internal\": node \"ip-10-0-129-205.ec2.internal\" not found"
Sep 17 10:53:25.237259 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.237239    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-205.ec2.internal"]
Sep 17 10:53:25.237333 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.237293    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 10:53:25.238653 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.238639    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 10:53:25.238716 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.238663    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 10:53:25.238716 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.238672    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasSufficientPID"
Sep 17 10:53:25.240732 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.240721    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 10:53:25.240881 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.240861    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.240936 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.240927    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 10:53:25.241585 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.241567    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 10:53:25.241658 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.241598    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 10:53:25.241658 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.241617    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 10:53:25.241658 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.241627    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 10:53:25.241658 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.241634    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasSufficientPID"
Sep 17 10:53:25.241658 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.241645    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasSufficientPID"
Sep 17 10:53:25.243685 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.243671    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.243747 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.243696    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 10:53:25.244479 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.244463    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 10:53:25.244558 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.244494    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 10:53:25.244558 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.244518    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeHasSufficientPID"
Sep 17 10:53:25.245309 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.245292    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-205.ec2.internal\" not found"
Sep 17 10:53:25.282616 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.282589    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-205.ec2.internal\" not found" node="ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.288430 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.288411    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-205.ec2.internal\" not found" node="ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.334771 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.334744    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e413a23ab9533a4b31565bf83d28336b-config\") pod \"kube-apiserver-proxy-ip-10-0-129-205.ec2.internal\" (UID: \"e413a23ab9533a4b31565bf83d28336b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.334859 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.334773    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bb68f00172f0a347af69ef95526d7c0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal\" (UID: \"bb68f00172f0a347af69ef95526d7c0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.334859 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.334814    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb68f00172f0a347af69ef95526d7c0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal\" (UID: \"bb68f00172f0a347af69ef95526d7c0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.345876 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.345853    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-205.ec2.internal\" not found"
Sep 17 10:53:25.435276 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.435250    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bb68f00172f0a347af69ef95526d7c0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal\" (UID: \"bb68f00172f0a347af69ef95526d7c0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.435383 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.435316    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bb68f00172f0a347af69ef95526d7c0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal\" (UID: \"bb68f00172f0a347af69ef95526d7c0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.435383 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.435353    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb68f00172f0a347af69ef95526d7c0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal\" (UID: \"bb68f00172f0a347af69ef95526d7c0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.435383 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.435378    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e413a23ab9533a4b31565bf83d28336b-config\") pod \"kube-apiserver-proxy-ip-10-0-129-205.ec2.internal\" (UID: \"e413a23ab9533a4b31565bf83d28336b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.435481 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.435422    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e413a23ab9533a4b31565bf83d28336b-config\") pod \"kube-apiserver-proxy-ip-10-0-129-205.ec2.internal\" (UID: \"e413a23ab9533a4b31565bf83d28336b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.435481 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.435437    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb68f00172f0a347af69ef95526d7c0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal\" (UID: \"bb68f00172f0a347af69ef95526d7c0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.446378 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.446357    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-205.ec2.internal\" not found"
Sep 17 10:53:25.547086 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.547065    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-205.ec2.internal\" not found"
Sep 17 10:53:25.585242 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.585214    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.591102 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.591088    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-205.ec2.internal"
Sep 17 10:53:25.647774 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.647749    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-205.ec2.internal\" not found"
Sep 17 10:53:25.748320 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.748297    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-205.ec2.internal\" not found"
Sep 17 10:53:25.848799 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.848736    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-205.ec2.internal\" not found"
Sep 17 10:53:25.949294 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:25.949267    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-205.ec2.internal\" not found"
Sep 17 10:53:25.957046 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.957028    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 10:53:25.967139 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.967117    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 10:53:25.967249 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.967230    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 10:53:25.967289 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:25.967268    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 10:53:26.050117 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:26.050094    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-205.ec2.internal\" not found"
Sep 17 10:53:26.064261 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:26.064231    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 10:48:25 +0000 UTC" deadline="2028-02-27 17:31:06.628473744 +0000 UTC"
Sep 17 10:53:26.064261 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:26.064257    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12678h37m40.56421991s"
Sep 17 10:53:26.151045 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:26.150882    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-205.ec2.internal\" not found"
Sep 17 10:53:26.213744 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:26.213703    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode413a23ab9533a4b31565bf83d28336b.slice/crio-4c52549cc60cd971778376c70d656c9dd828919eb19cb1b951d807091607adea WatchSource:0}: Error finding container 4c52549cc60cd971778376c70d656c9dd828919eb19cb1b951d807091607adea: Status 404 returned error can't find the container with id 4c52549cc60cd971778376c70d656c9dd828919eb19cb1b951d807091607adea
Sep 17 10:53:26.217724 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:26.217711    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 10:53:26.251984 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:26.251965    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-205.ec2.internal\" not found"
Sep 17 10:53:26.352506 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:26.352463    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-205.ec2.internal\" not found"
Sep 17 10:53:26.407191 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:26.407163    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 10:53:26.413418 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:26.413400    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb68f00172f0a347af69ef95526d7c0e.slice/crio-b78ff23bbeab5d8e5fdec5d69a945f3646ce45dc17c8208b3dcfee1a3e0811e4 WatchSource:0}: Error finding container b78ff23bbeab5d8e5fdec5d69a945f3646ce45dc17c8208b3dcfee1a3e0811e4: Status 404 returned error can't find the container with id b78ff23bbeab5d8e5fdec5d69a945f3646ce45dc17c8208b3dcfee1a3e0811e4
Sep 17 10:53:26.453359 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:26.453337    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-205.ec2.internal\" not found"
Sep 17 10:53:26.530441 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:26.530424    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 10:53:26.532960 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:26.532947    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal"
Sep 17 10:53:26.563049 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:26.563033    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 10:53:26.564112 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:26.564099    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-205.ec2.internal"
Sep 17 10:53:26.579287 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:26.579270    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 10:53:27.015455 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.015432    2564 apiserver.go:52] "Watching apiserver"
Sep 17 10:53:27.021418 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.021395    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 10:53:27.024228 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.024197    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-205.ec2.internal","openshift-cluster-node-tuning-operator/tuned-d7dvr","openshift-image-registry/node-ca-rdz5r","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal","openshift-multus/multus-additional-cni-plugins-thp9w","openshift-multus/multus-wsj8z","openshift-multus/network-metrics-daemon-g7wtx","openshift-network-diagnostics/network-check-target-fnw2s","kube-system/global-pull-secret-syncer-ddcnf","kube-system/konnectivity-agent-fdnxp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4","openshift-dns/node-resolver-phs4b","openshift-network-operator/iptables-alerter-jnlqq","openshift-ovn-kubernetes/ovnkube-node-c559v"]
Sep 17 10:53:27.026893 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.026870    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.029302 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.029271    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 10:53:27.029395 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.029370    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.029486 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.029459    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 10:53:27.029671 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.029656    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 10:53:27.029716 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.029665    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 10:53:27.029716 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.029689    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 10:53:27.029854 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.029841    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fmzpj\""
Sep 17 10:53:27.032583 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.032557    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gxh4j\""
Sep 17 10:53:27.032687 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.032639    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-phs4b"
Sep 17 10:53:27.033825 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.033781    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 10:53:27.034615 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.034592    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 10:53:27.034729 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.034619    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 10:53:27.034892 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.034873    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 10:53:27.035162 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.035140    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.035253 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.035207    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7g47n\""
Sep 17 10:53:27.035706 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.035690    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 10:53:27.039179 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.038396    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fdnxp"
Sep 17 10:53:27.039179 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.038789    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 10:53:27.039179 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.038887    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-98pzb\""
Sep 17 10:53:27.043125 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.043104    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5brkj\""
Sep 17 10:53:27.043332 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.043315    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 10:53:27.043418 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.043352    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:27.043418 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.043388    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 10:53:27.043495 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.043408    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:27.044852 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.044793    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-run-netns\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.044852 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.044832    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-multus-cni-dir\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.045087 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.044866    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-var-lib-cni-multus\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.045087 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.044897    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cff16177-6fce-4305-8de0-742e7c49db64-cni-binary-copy\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.045087 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.044946    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-lib-modules\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.045087 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.044981    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-var-lib-kubelet\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.045087 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045004    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-system-cni-dir\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.045087 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045029    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-multus-socket-dir-parent\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.045087 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045052    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-multus-conf-dir\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.045087 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045077    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6pk9\" (UniqueName: \"kubernetes.io/projected/cff16177-6fce-4305-8de0-742e7c49db64-kube-api-access-d6pk9\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.045488 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045101    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-modprobe-d\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.045488 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045127    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ae000258-0b0b-4b77-a6d7-85091d242316-etc-tuned\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.045488 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045160    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztfvj\" (UniqueName: \"kubernetes.io/projected/ae000258-0b0b-4b77-a6d7-85091d242316-kube-api-access-ztfvj\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.045488 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045184    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a02f264e-0ec3-4ea3-b2a8-aca37526f1c3-tmp-dir\") pod \"node-resolver-phs4b\" (UID: \"a02f264e-0ec3-4ea3-b2a8-aca37526f1c3\") " pod="openshift-dns/node-resolver-phs4b"
Sep 17 10:53:27.045488 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045207    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgcd4\" (UniqueName: \"kubernetes.io/projected/a02f264e-0ec3-4ea3-b2a8-aca37526f1c3-kube-api-access-pgcd4\") pod \"node-resolver-phs4b\" (UID: \"a02f264e-0ec3-4ea3-b2a8-aca37526f1c3\") " pod="openshift-dns/node-resolver-phs4b"
Sep 17 10:53:27.045488 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045230    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5db558b4-e831-4239-9486-ca2420b01bd7-cni-binary-copy\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.045488 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045271    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-var-lib-kubelet\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.045488 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045365    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-hostroot\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.045488 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045392    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-etc-kubernetes\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.045488 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045418    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cff16177-6fce-4305-8de0-742e7c49db64-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.045488 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045443    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-run-multus-certs\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.045488 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045486    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-sysctl-conf\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045509    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-sys\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045534    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cff16177-6fce-4305-8de0-742e7c49db64-tuning-conf-dir\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045559    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-run-k8s-cni-cncf-io\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045585    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a02f264e-0ec3-4ea3-b2a8-aca37526f1c3-hosts-file\") pod \"node-resolver-phs4b\" (UID: \"a02f264e-0ec3-4ea3-b2a8-aca37526f1c3\") " pod="openshift-dns/node-resolver-phs4b"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045609    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-var-lib-cni-bin\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5db558b4-e831-4239-9486-ca2420b01bd7-multus-daemon-config\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045696    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nlwb\" (UniqueName: \"kubernetes.io/projected/5db558b4-e831-4239-9486-ca2420b01bd7-kube-api-access-9nlwb\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045721    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cff16177-6fce-4305-8de0-742e7c49db64-cnibin\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045748    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-kubernetes\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045771    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-run\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045793    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-host\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045817    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cff16177-6fce-4305-8de0-742e7c49db64-system-cni-dir\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045841    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-sysconfig\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045877    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae000258-0b0b-4b77-a6d7-85091d242316-tmp\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045945    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-cnibin\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.045985    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cff16177-6fce-4305-8de0-742e7c49db64-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.046084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.046009    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:27.046727 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.046016    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-sysctl-d\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.046727 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.046040    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-systemd\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.046727 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.046068    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:27.048113 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.048088    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 10:53:27.048418 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.048393    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:27.048506 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.048468    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:27.050950 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.050930    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.054421 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.054397    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rdz5r"
Sep 17 10:53:27.055199 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.055178    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 10:53:27.055299 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.055189    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 10:53:27.055458 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.055364    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 10:53:27.055663 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.055645    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-96p5l\""
Sep 17 10:53:27.057266 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.057230    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jnlqq"
Sep 17 10:53:27.058521 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.058501    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 10:53:27.058793 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.058773    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 10:53:27.058883 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.058822    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rb9z7\""
Sep 17 10:53:27.059190 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.059172    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 10:53:27.059871 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.059854    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.069215 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.069191    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 10:53:27.069315 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.069235    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 10:53:27.069376 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.069340    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 10:53:27.069505 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.069479    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 10:53:27.069621 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.069606    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 10:53:27.069693 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.069680    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 10:53:27.069750 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.069708    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 10:53:27.070109 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.069894    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jbb5p\""
Sep 17 10:53:27.070109 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.069935    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kjhsk\""
Sep 17 10:53:27.070109 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.070016    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 10:53:27.070109 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.070028    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 10:53:27.084435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.084413    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4b759"
Sep 17 10:53:27.094413 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.094391    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4b759"
Sep 17 10:53:27.135327 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.135308    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 10:53:27.142024 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.141963    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal" event={"ID":"bb68f00172f0a347af69ef95526d7c0e","Type":"ContainerStarted","Data":"b78ff23bbeab5d8e5fdec5d69a945f3646ce45dc17c8208b3dcfee1a3e0811e4"}
Sep 17 10:53:27.150211 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150171    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-205.ec2.internal" event={"ID":"e413a23ab9533a4b31565bf83d28336b","Type":"ContainerStarted","Data":"4c52549cc60cd971778376c70d656c9dd828919eb19cb1b951d807091607adea"}
Sep 17 10:53:27.150460 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150439    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-run-netns\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.150536 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150476    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a02f264e-0ec3-4ea3-b2a8-aca37526f1c3-tmp-dir\") pod \"node-resolver-phs4b\" (UID: \"a02f264e-0ec3-4ea3-b2a8-aca37526f1c3\") " pod="openshift-dns/node-resolver-phs4b"
Sep 17 10:53:27.150536 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150507    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgcd4\" (UniqueName: \"kubernetes.io/projected/a02f264e-0ec3-4ea3-b2a8-aca37526f1c3-kube-api-access-pgcd4\") pod \"node-resolver-phs4b\" (UID: \"a02f264e-0ec3-4ea3-b2a8-aca37526f1c3\") " pod="openshift-dns/node-resolver-phs4b"
Sep 17 10:53:27.150635 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150535    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-etc-kubernetes\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.150635 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150569    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cff16177-6fce-4305-8de0-742e7c49db64-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.150635 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150603    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-sysctl-conf\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.150635 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150629    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-run-ovn-kubernetes\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.150819 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150649    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-cni-netd\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.150819 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150666    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-sys\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.150819 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150694    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-var-lib-openvswitch\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.150819 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150714    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-etc-openvswitch\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.150819 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150732    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-run-openvswitch\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.150819 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150750    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.150819 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150803    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cff16177-6fce-4305-8de0-742e7c49db64-tuning-conf-dir\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.151184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150837    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-run-k8s-cni-cncf-io\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.151184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150872    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d1fe18b-e374-4453-a560-3aa3245d100b-konnectivity-ca\") pod \"konnectivity-agent-fdnxp\" (UID: \"3d1fe18b-e374-4453-a560-3aa3245d100b\") " pod="kube-system/konnectivity-agent-fdnxp"
Sep 17 10:53:27.151184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150905    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/909dc7e0-fcc5-4736-8874-8d5206f3f741-dbus\") pod \"global-pull-secret-syncer-ddcnf\" (UID: \"909dc7e0-fcc5-4736-8874-8d5206f3f741\") " pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:27.151184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150928    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-sys-fs\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.151184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150944    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d25df7be-fd4f-4a61-8313-ebbe3f18fcd8-iptables-alerter-script\") pod \"iptables-alerter-jnlqq\" (UID: \"d25df7be-fd4f-4a61-8313-ebbe3f18fcd8\") " pod="openshift-network-operator/iptables-alerter-jnlqq"
Sep 17 10:53:27.151184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.150993    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00db4623-b291-4c7e-8674-1eb725b6584d-ovn-node-metrics-cert\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.151184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151035    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-var-lib-cni-bin\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.151184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151066    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-kubernetes\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.151184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151086    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-run-k8s-cni-cncf-io\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.151184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151097    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-run\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.151184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151171    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-host\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151206    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-registration-dir\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151239    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-run-systemd\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151239    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-run\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151334    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-run-ovn\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151339    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-sysctl-conf\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151372    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cff16177-6fce-4305-8de0-742e7c49db64-system-cni-dir\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151370    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-sys\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151408    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wngvt\" (UniqueName: \"kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt\") pod \"network-check-target-fnw2s\" (UID: \"82f7e981-33bb-4353-99bc-1500f39cd125\") " pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151460    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-host\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151492    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151494    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cff16177-6fce-4305-8de0-742e7c49db64-tuning-conf-dir\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151516    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-etc-kubernetes\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151527    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00db4623-b291-4c7e-8674-1eb725b6584d-ovnkube-script-lib\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151570    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-cnibin\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151585    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-var-lib-cni-bin\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151606    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cff16177-6fce-4305-8de0-742e7c49db64-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.151734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151644    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-systemd\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151711    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-kubernetes\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151767    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cff16177-6fce-4305-8de0-742e7c49db64-system-cni-dir\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151771    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-cnibin\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151808    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-systemd\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151712    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v2sb\" (UniqueName: \"kubernetes.io/projected/00db4623-b291-4c7e-8674-1eb725b6584d-kube-api-access-2v2sb\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151844    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a02f264e-0ec3-4ea3-b2a8-aca37526f1c3-tmp-dir\") pod \"node-resolver-phs4b\" (UID: \"a02f264e-0ec3-4ea3-b2a8-aca37526f1c3\") " pod="openshift-dns/node-resolver-phs4b"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151885    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-multus-cni-dir\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151951    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-multus-cni-dir\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151954    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-var-lib-cni-multus\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.151997    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cff16177-6fce-4305-8de0-742e7c49db64-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152037    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-var-lib-cni-multus\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152062    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cff16177-6fce-4305-8de0-742e7c49db64-cni-binary-copy\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152134    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-lib-modules\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152168    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-modprobe-d\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152230    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74rrj\" (UniqueName: \"kubernetes.io/projected/d25df7be-fd4f-4a61-8313-ebbe3f18fcd8-kube-api-access-74rrj\") pod \"iptables-alerter-jnlqq\" (UID: \"d25df7be-fd4f-4a61-8313-ebbe3f18fcd8\") " pod="openshift-network-operator/iptables-alerter-jnlqq"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152263    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-cni-bin\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.152561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152266    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-modprobe-d\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152291    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5db558b4-e831-4239-9486-ca2420b01bd7-cni-binary-copy\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152312    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-var-lib-kubelet\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152331    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-hostroot\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152355    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cff16177-6fce-4305-8de0-742e7c49db64-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152371    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjpdq\" (UniqueName: \"kubernetes.io/projected/c0028c99-f81b-4054-9098-152b859dd374-kube-api-access-bjpdq\") pod \"network-metrics-daemon-g7wtx\" (UID: \"c0028c99-f81b-4054-9098-152b859dd374\") " pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152407    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-node-log\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152410    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-hostroot\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152427    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-lib-modules\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152461    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-run-multus-certs\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152500    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-etc-selinux\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152537    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-kubelet\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152596    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cff16177-6fce-4305-8de0-742e7c49db64-cni-binary-copy\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152597    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a02f264e-0ec3-4ea3-b2a8-aca37526f1c3-hosts-file\") pod \"node-resolver-phs4b\" (UID: \"a02f264e-0ec3-4ea3-b2a8-aca37526f1c3\") " pod="openshift-dns/node-resolver-phs4b"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152648    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d1fe18b-e374-4453-a560-3aa3245d100b-agent-certs\") pod \"konnectivity-agent-fdnxp\" (UID: \"3d1fe18b-e374-4453-a560-3aa3245d100b\") " pod="kube-system/konnectivity-agent-fdnxp"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152651    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a02f264e-0ec3-4ea3-b2a8-aca37526f1c3-hosts-file\") pod \"node-resolver-phs4b\" (UID: \"a02f264e-0ec3-4ea3-b2a8-aca37526f1c3\") " pod="openshift-dns/node-resolver-phs4b"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152682    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-systemd-units\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.153375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152740    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-slash\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152744    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-run-multus-certs\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152772    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00db4623-b291-4c7e-8674-1eb725b6584d-ovnkube-config\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152810    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5db558b4-e831-4239-9486-ca2420b01bd7-multus-daemon-config\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152840    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nlwb\" (UniqueName: \"kubernetes.io/projected/5db558b4-e831-4239-9486-ca2420b01bd7-kube-api-access-9nlwb\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152862    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-var-lib-kubelet\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152942    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cff16177-6fce-4305-8de0-742e7c49db64-cnibin\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152982    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scz98\" (UniqueName: \"kubernetes.io/projected/76f4d810-0354-43d9-871d-f9d94b5d8a98-kube-api-access-scz98\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.152953    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5db558b4-e831-4239-9486-ca2420b01bd7-cni-binary-copy\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153028    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f7775776-e097-4507-9a6a-5d21c14ebe07-host\") pod \"node-ca-rdz5r\" (UID: \"f7775776-e097-4507-9a6a-5d21c14ebe07\") " pod="openshift-image-registry/node-ca-rdz5r"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153060    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7jxs\" (UniqueName: \"kubernetes.io/projected/f7775776-e097-4507-9a6a-5d21c14ebe07-kube-api-access-n7jxs\") pod \"node-ca-rdz5r\" (UID: \"f7775776-e097-4507-9a6a-5d21c14ebe07\") " pod="openshift-image-registry/node-ca-rdz5r"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153092    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-log-socket\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153150    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-sysconfig\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153201    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cff16177-6fce-4305-8de0-742e7c49db64-cnibin\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153241    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae000258-0b0b-4b77-a6d7-85091d242316-tmp\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153293    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00db4623-b291-4c7e-8674-1eb725b6584d-env-overrides\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153336    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-sysctl-d\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.154203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153387    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs\") pod \"network-metrics-daemon-g7wtx\" (UID: \"c0028c99-f81b-4054-9098-152b859dd374\") " pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153444    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/909dc7e0-fcc5-4736-8874-8d5206f3f741-kubelet-config\") pod \"global-pull-secret-syncer-ddcnf\" (UID: \"909dc7e0-fcc5-4736-8874-8d5206f3f741\") " pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153457    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5db558b4-e831-4239-9486-ca2420b01bd7-multus-daemon-config\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153476    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret\") pod \"global-pull-secret-syncer-ddcnf\" (UID: \"909dc7e0-fcc5-4736-8874-8d5206f3f741\") " pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153533    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-socket-dir\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153593    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-run-netns\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153627    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-var-lib-kubelet\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153657    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-device-dir\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153704    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-sysctl-d\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153714    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f7775776-e097-4507-9a6a-5d21c14ebe07-serviceca\") pod \"node-ca-rdz5r\" (UID: \"f7775776-e097-4507-9a6a-5d21c14ebe07\") " pod="openshift-image-registry/node-ca-rdz5r"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153788    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-system-cni-dir\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153832    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-multus-socket-dir-parent\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153837    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-etc-sysconfig\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153853    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-host-run-netns\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153864    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-multus-conf-dir\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153900    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6pk9\" (UniqueName: \"kubernetes.io/projected/cff16177-6fce-4305-8de0-742e7c49db64-kube-api-access-d6pk9\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153904    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae000258-0b0b-4b77-a6d7-85091d242316-var-lib-kubelet\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153910    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 10:53:27.154918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153951    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-multus-socket-dir-parent\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.155734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.153995    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ae000258-0b0b-4b77-a6d7-85091d242316-etc-tuned\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.155734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.154004    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-system-cni-dir\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.155734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.154039    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5db558b4-e831-4239-9486-ca2420b01bd7-multus-conf-dir\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.155734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.154059    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztfvj\" (UniqueName: \"kubernetes.io/projected/ae000258-0b0b-4b77-a6d7-85091d242316-kube-api-access-ztfvj\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.155734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.154218    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d25df7be-fd4f-4a61-8313-ebbe3f18fcd8-host-slash\") pod \"iptables-alerter-jnlqq\" (UID: \"d25df7be-fd4f-4a61-8313-ebbe3f18fcd8\") " pod="openshift-network-operator/iptables-alerter-jnlqq"
Sep 17 10:53:27.158458 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.158435    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ae000258-0b0b-4b77-a6d7-85091d242316-etc-tuned\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.158458 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.158450    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae000258-0b0b-4b77-a6d7-85091d242316-tmp\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.160059 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.159994    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgcd4\" (UniqueName: \"kubernetes.io/projected/a02f264e-0ec3-4ea3-b2a8-aca37526f1c3-kube-api-access-pgcd4\") pod \"node-resolver-phs4b\" (UID: \"a02f264e-0ec3-4ea3-b2a8-aca37526f1c3\") " pod="openshift-dns/node-resolver-phs4b"
Sep 17 10:53:27.162264 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.162220    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nlwb\" (UniqueName: \"kubernetes.io/projected/5db558b4-e831-4239-9486-ca2420b01bd7-kube-api-access-9nlwb\") pod \"multus-wsj8z\" (UID: \"5db558b4-e831-4239-9486-ca2420b01bd7\") " pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.162743 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.162704    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6pk9\" (UniqueName: \"kubernetes.io/projected/cff16177-6fce-4305-8de0-742e7c49db64-kube-api-access-d6pk9\") pod \"multus-additional-cni-plugins-thp9w\" (UID: \"cff16177-6fce-4305-8de0-742e7c49db64\") " pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.163655 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.163627    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztfvj\" (UniqueName: \"kubernetes.io/projected/ae000258-0b0b-4b77-a6d7-85091d242316-kube-api-access-ztfvj\") pod \"tuned-d7dvr\" (UID: \"ae000258-0b0b-4b77-a6d7-85091d242316\") " pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.254850 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.254815    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scz98\" (UniqueName: \"kubernetes.io/projected/76f4d810-0354-43d9-871d-f9d94b5d8a98-kube-api-access-scz98\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.255003 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.254868    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f7775776-e097-4507-9a6a-5d21c14ebe07-host\") pod \"node-ca-rdz5r\" (UID: \"f7775776-e097-4507-9a6a-5d21c14ebe07\") " pod="openshift-image-registry/node-ca-rdz5r"
Sep 17 10:53:27.255003 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.254886    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7jxs\" (UniqueName: \"kubernetes.io/projected/f7775776-e097-4507-9a6a-5d21c14ebe07-kube-api-access-n7jxs\") pod \"node-ca-rdz5r\" (UID: \"f7775776-e097-4507-9a6a-5d21c14ebe07\") " pod="openshift-image-registry/node-ca-rdz5r"
Sep 17 10:53:27.255003 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.254902    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-log-socket\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.255003 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.254941    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00db4623-b291-4c7e-8674-1eb725b6584d-env-overrides\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.255003 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.254972    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f7775776-e097-4507-9a6a-5d21c14ebe07-host\") pod \"node-ca-rdz5r\" (UID: \"f7775776-e097-4507-9a6a-5d21c14ebe07\") " pod="openshift-image-registry/node-ca-rdz5r"
Sep 17 10:53:27.255253 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255003    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-log-socket\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.255253 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255043    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs\") pod \"network-metrics-daemon-g7wtx\" (UID: \"c0028c99-f81b-4054-9098-152b859dd374\") " pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:27.255253 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255092    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/909dc7e0-fcc5-4736-8874-8d5206f3f741-kubelet-config\") pod \"global-pull-secret-syncer-ddcnf\" (UID: \"909dc7e0-fcc5-4736-8874-8d5206f3f741\") " pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:27.255253 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255119    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret\") pod \"global-pull-secret-syncer-ddcnf\" (UID: \"909dc7e0-fcc5-4736-8874-8d5206f3f741\") " pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:27.255253 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255150    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-socket-dir\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.255253 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255179    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-device-dir\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.255253 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255195    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/909dc7e0-fcc5-4736-8874-8d5206f3f741-kubelet-config\") pod \"global-pull-secret-syncer-ddcnf\" (UID: \"909dc7e0-fcc5-4736-8874-8d5206f3f741\") " pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:27.255253 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.255204    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 10:53:27.255253 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.255232    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 10:53:27.255624 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255205    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f7775776-e097-4507-9a6a-5d21c14ebe07-serviceca\") pod \"node-ca-rdz5r\" (UID: \"f7775776-e097-4507-9a6a-5d21c14ebe07\") " pod="openshift-image-registry/node-ca-rdz5r"
Sep 17 10:53:27.255624 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.255301    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret podName:909dc7e0-fcc5-4736-8874-8d5206f3f741 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:27.755278028 +0000 UTC m=+3.071641181 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret") pod "global-pull-secret-syncer-ddcnf" (UID: "909dc7e0-fcc5-4736-8874-8d5206f3f741") : object "kube-system"/"original-pull-secret" not registered
Sep 17 10:53:27.255624 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255306    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-socket-dir\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.255624 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255339    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-device-dir\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.255624 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.255350    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs podName:c0028c99-f81b-4054-9098-152b859dd374 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:27.755330427 +0000 UTC m=+3.071693568 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs") pod "network-metrics-daemon-g7wtx" (UID: "c0028c99-f81b-4054-9098-152b859dd374") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 10:53:27.255624 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255391    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d25df7be-fd4f-4a61-8313-ebbe3f18fcd8-host-slash\") pod \"iptables-alerter-jnlqq\" (UID: \"d25df7be-fd4f-4a61-8313-ebbe3f18fcd8\") " pod="openshift-network-operator/iptables-alerter-jnlqq"
Sep 17 10:53:27.255624 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255419    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-run-netns\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.255624 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255451    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d25df7be-fd4f-4a61-8313-ebbe3f18fcd8-host-slash\") pod \"iptables-alerter-jnlqq\" (UID: \"d25df7be-fd4f-4a61-8313-ebbe3f18fcd8\") " pod="openshift-network-operator/iptables-alerter-jnlqq"
Sep 17 10:53:27.255624 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255455    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-run-ovn-kubernetes\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.255624 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-cni-netd\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.255624 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255489    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/00db4623-b291-4c7e-8674-1eb725b6584d-env-overrides\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.255624 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255498    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-run-netns\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.255624 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255508    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-var-lib-openvswitch\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.255624 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255483    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-run-ovn-kubernetes\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.255624 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255504    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-cni-netd\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.255624 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255547    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-etc-openvswitch\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255568    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-run-openvswitch\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255584    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255588    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-etc-openvswitch\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255609    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d1fe18b-e374-4453-a560-3aa3245d100b-konnectivity-ca\") pod \"konnectivity-agent-fdnxp\" (UID: \"3d1fe18b-e374-4453-a560-3aa3245d100b\") " pod="kube-system/konnectivity-agent-fdnxp"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255618    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f7775776-e097-4507-9a6a-5d21c14ebe07-serviceca\") pod \"node-ca-rdz5r\" (UID: \"f7775776-e097-4507-9a6a-5d21c14ebe07\") " pod="openshift-image-registry/node-ca-rdz5r"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255549    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-var-lib-openvswitch\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255636    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255634    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/909dc7e0-fcc5-4736-8874-8d5206f3f741-dbus\") pod \"global-pull-secret-syncer-ddcnf\" (UID: \"909dc7e0-fcc5-4736-8874-8d5206f3f741\") " pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255665    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-run-openvswitch\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255682    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-sys-fs\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255705    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d25df7be-fd4f-4a61-8313-ebbe3f18fcd8-iptables-alerter-script\") pod \"iptables-alerter-jnlqq\" (UID: \"d25df7be-fd4f-4a61-8313-ebbe3f18fcd8\") " pod="openshift-network-operator/iptables-alerter-jnlqq"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255730    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00db4623-b291-4c7e-8674-1eb725b6584d-ovn-node-metrics-cert\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255745    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-sys-fs\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-registration-dir\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255778    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-run-systemd\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255792    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/909dc7e0-fcc5-4736-8874-8d5206f3f741-dbus\") pod \"global-pull-secret-syncer-ddcnf\" (UID: \"909dc7e0-fcc5-4736-8874-8d5206f3f741\") " pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:27.256318 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255804    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-run-ovn\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255854    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-registration-dir\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255861    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wngvt\" (UniqueName: \"kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt\") pod \"network-check-target-fnw2s\" (UID: \"82f7e981-33bb-4353-99bc-1500f39cd125\") " pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255905    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.255958    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00db4623-b291-4c7e-8674-1eb725b6584d-ovnkube-script-lib\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256009    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v2sb\" (UniqueName: \"kubernetes.io/projected/00db4623-b291-4c7e-8674-1eb725b6584d-kube-api-access-2v2sb\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256024    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-run-ovn\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74rrj\" (UniqueName: \"kubernetes.io/projected/d25df7be-fd4f-4a61-8313-ebbe3f18fcd8-kube-api-access-74rrj\") pod \"iptables-alerter-jnlqq\" (UID: \"d25df7be-fd4f-4a61-8313-ebbe3f18fcd8\") " pod="openshift-network-operator/iptables-alerter-jnlqq"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256075    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-run-systemd\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256084    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-cni-bin\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256108    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjpdq\" (UniqueName: \"kubernetes.io/projected/c0028c99-f81b-4054-9098-152b859dd374-kube-api-access-bjpdq\") pod \"network-metrics-daemon-g7wtx\" (UID: \"c0028c99-f81b-4054-9098-152b859dd374\") " pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256123    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-node-log\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256139    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-etc-selinux\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256161    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-kubelet\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256186    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d1fe18b-e374-4453-a560-3aa3245d100b-agent-certs\") pod \"konnectivity-agent-fdnxp\" (UID: \"3d1fe18b-e374-4453-a560-3aa3245d100b\") " pod="kube-system/konnectivity-agent-fdnxp"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256211    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-systemd-units\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256217    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-cni-bin\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256234    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-slash\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257740 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256260    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00db4623-b291-4c7e-8674-1eb725b6584d-ovnkube-config\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257740 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256318    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-etc-selinux\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.257740 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256427    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-node-log\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257740 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256439    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/76f4d810-0354-43d9-871d-f9d94b5d8a98-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.257740 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256492    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-systemd-units\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257740 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256495    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-slash\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257740 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256680    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d1fe18b-e374-4453-a560-3aa3245d100b-konnectivity-ca\") pod \"konnectivity-agent-fdnxp\" (UID: \"3d1fe18b-e374-4453-a560-3aa3245d100b\") " pod="kube-system/konnectivity-agent-fdnxp"
Sep 17 10:53:27.257740 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.256955    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d25df7be-fd4f-4a61-8313-ebbe3f18fcd8-iptables-alerter-script\") pod \"iptables-alerter-jnlqq\" (UID: \"d25df7be-fd4f-4a61-8313-ebbe3f18fcd8\") " pod="openshift-network-operator/iptables-alerter-jnlqq"
Sep 17 10:53:27.257740 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.257070    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/00db4623-b291-4c7e-8674-1eb725b6584d-host-kubelet\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257740 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.257074    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/00db4623-b291-4c7e-8674-1eb725b6584d-ovnkube-script-lib\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.257740 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.257676    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/00db4623-b291-4c7e-8674-1eb725b6584d-ovnkube-config\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.259169 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.259142    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/00db4623-b291-4c7e-8674-1eb725b6584d-ovn-node-metrics-cert\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.259563 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.259539    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d1fe18b-e374-4453-a560-3aa3245d100b-agent-certs\") pod \"konnectivity-agent-fdnxp\" (UID: \"3d1fe18b-e374-4453-a560-3aa3245d100b\") " pod="kube-system/konnectivity-agent-fdnxp"
Sep 17 10:53:27.275181 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.275114    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 10:53:27.275181 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.275140    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 10:53:27.275181 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.275155    2564 projected.go:196] Error preparing data for projected volume kube-api-access-wngvt for pod openshift-network-diagnostics/network-check-target-fnw2s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 10:53:27.275379 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.275216    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt podName:82f7e981-33bb-4353-99bc-1500f39cd125 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:27.775199132 +0000 UTC m=+3.091562281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wngvt" (UniqueName: "kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt") pod "network-check-target-fnw2s" (UID: "82f7e981-33bb-4353-99bc-1500f39cd125") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 10:53:27.275914 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.275871    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74rrj\" (UniqueName: \"kubernetes.io/projected/d25df7be-fd4f-4a61-8313-ebbe3f18fcd8-kube-api-access-74rrj\") pod \"iptables-alerter-jnlqq\" (UID: \"d25df7be-fd4f-4a61-8313-ebbe3f18fcd8\") " pod="openshift-network-operator/iptables-alerter-jnlqq"
Sep 17 10:53:27.277649 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.277625    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7jxs\" (UniqueName: \"kubernetes.io/projected/f7775776-e097-4507-9a6a-5d21c14ebe07-kube-api-access-n7jxs\") pod \"node-ca-rdz5r\" (UID: \"f7775776-e097-4507-9a6a-5d21c14ebe07\") " pod="openshift-image-registry/node-ca-rdz5r"
Sep 17 10:53:27.278704 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.278681    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scz98\" (UniqueName: \"kubernetes.io/projected/76f4d810-0354-43d9-871d-f9d94b5d8a98-kube-api-access-scz98\") pod \"aws-ebs-csi-driver-node-2xpk4\" (UID: \"76f4d810-0354-43d9-871d-f9d94b5d8a98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.279183 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.279165    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v2sb\" (UniqueName: \"kubernetes.io/projected/00db4623-b291-4c7e-8674-1eb725b6584d-kube-api-access-2v2sb\") pod \"ovnkube-node-c559v\" (UID: \"00db4623-b291-4c7e-8674-1eb725b6584d\") " pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.279660 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.279639    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjpdq\" (UniqueName: \"kubernetes.io/projected/c0028c99-f81b-4054-9098-152b859dd374-kube-api-access-bjpdq\") pod \"network-metrics-daemon-g7wtx\" (UID: \"c0028c99-f81b-4054-9098-152b859dd374\") " pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:27.290562 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.290541    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 10:53:27.342142 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.342115    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-thp9w"
Sep 17 10:53:27.351010 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.350985    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d7dvr"
Sep 17 10:53:27.360240 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.360218    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-phs4b"
Sep 17 10:53:27.367140 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.367121    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wsj8z"
Sep 17 10:53:27.375903 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.375882    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fdnxp"
Sep 17 10:53:27.383797 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.383771    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4"
Sep 17 10:53:27.392646 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.392628    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rdz5r"
Sep 17 10:53:27.400458 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.400420    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jnlqq"
Sep 17 10:53:27.407454 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.407438    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:27.760890 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.760860    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs\") pod \"network-metrics-daemon-g7wtx\" (UID: \"c0028c99-f81b-4054-9098-152b859dd374\") " pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:27.760890 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.760895    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret\") pod \"global-pull-secret-syncer-ddcnf\" (UID: \"909dc7e0-fcc5-4736-8874-8d5206f3f741\") " pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:27.761119 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.761054    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 10:53:27.761119 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.761112    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs podName:c0028c99-f81b-4054-9098-152b859dd374 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:28.761098746 +0000 UTC m=+4.077461894 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs") pod "network-metrics-daemon-g7wtx" (UID: "c0028c99-f81b-4054-9098-152b859dd374") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 10:53:27.761191 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.761054    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 10:53:27.761191 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.761176    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret podName:909dc7e0-fcc5-4736-8874-8d5206f3f741 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:28.761163448 +0000 UTC m=+4.077526593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret") pod "global-pull-secret-syncer-ddcnf" (UID: "909dc7e0-fcc5-4736-8874-8d5206f3f741") : object "kube-system"/"original-pull-secret" not registered
Sep 17 10:53:27.803691 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:27.803516    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd25df7be_fd4f_4a61_8313_ebbe3f18fcd8.slice/crio-2827156aba400f3fbec6d1308d33b2a1836fcfeb43e1b27e3beea3dbbd6d0712 WatchSource:0}: Error finding container 2827156aba400f3fbec6d1308d33b2a1836fcfeb43e1b27e3beea3dbbd6d0712: Status 404 returned error can't find the container with id 2827156aba400f3fbec6d1308d33b2a1836fcfeb43e1b27e3beea3dbbd6d0712
Sep 17 10:53:27.805284 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:27.805107    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7775776_e097_4507_9a6a_5d21c14ebe07.slice/crio-992715be0aaacc105fa82e5a25b7074a0a8f0340d85ea7acfc640e474a6c3349 WatchSource:0}: Error finding container 992715be0aaacc105fa82e5a25b7074a0a8f0340d85ea7acfc640e474a6c3349: Status 404 returned error can't find the container with id 992715be0aaacc105fa82e5a25b7074a0a8f0340d85ea7acfc640e474a6c3349
Sep 17 10:53:27.809517 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:27.808962    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00db4623_b291_4c7e_8674_1eb725b6584d.slice/crio-50f6247d0f7af1907655cb4ecc8dad430804a1d8171a3c8912e003f6f7f0d849 WatchSource:0}: Error finding container 50f6247d0f7af1907655cb4ecc8dad430804a1d8171a3c8912e003f6f7f0d849: Status 404 returned error can't find the container with id 50f6247d0f7af1907655cb4ecc8dad430804a1d8171a3c8912e003f6f7f0d849
Sep 17 10:53:27.809873 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:27.809771    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae000258_0b0b_4b77_a6d7_85091d242316.slice/crio-febfef32e9b4ea02e74d566b8224d194c208d714542699bbf107a871abc1916c WatchSource:0}: Error finding container febfef32e9b4ea02e74d566b8224d194c208d714542699bbf107a871abc1916c: Status 404 returned error can't find the container with id febfef32e9b4ea02e74d566b8224d194c208d714542699bbf107a871abc1916c
Sep 17 10:53:27.810551 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:27.810531    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda02f264e_0ec3_4ea3_b2a8_aca37526f1c3.slice/crio-1019f59a1f27f6a9047a613c075c150ed8858bf513c874a53b0de3c30b6783be WatchSource:0}: Error finding container 1019f59a1f27f6a9047a613c075c150ed8858bf513c874a53b0de3c30b6783be: Status 404 returned error can't find the container with id 1019f59a1f27f6a9047a613c075c150ed8858bf513c874a53b0de3c30b6783be
Sep 17 10:53:27.811664 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:27.811638    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f4d810_0354_43d9_871d_f9d94b5d8a98.slice/crio-6649044f13a349a963b321e75928146f908b4713c36ca870ad53b015ca69c917 WatchSource:0}: Error finding container 6649044f13a349a963b321e75928146f908b4713c36ca870ad53b015ca69c917: Status 404 returned error can't find the container with id 6649044f13a349a963b321e75928146f908b4713c36ca870ad53b015ca69c917
Sep 17 10:53:27.813744 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:27.813648    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcff16177_6fce_4305_8de0_742e7c49db64.slice/crio-d9c5b5df05748c345c5c324f359ef1a04bf9e3b0a60d0cc6b78152fdae4c1481 WatchSource:0}: Error finding container d9c5b5df05748c345c5c324f359ef1a04bf9e3b0a60d0cc6b78152fdae4c1481: Status 404 returned error can't find the container with id d9c5b5df05748c345c5c324f359ef1a04bf9e3b0a60d0cc6b78152fdae4c1481
Sep 17 10:53:27.814651 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:27.814629    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5db558b4_e831_4239_9486_ca2420b01bd7.slice/crio-055042e5b739245399f2a9d0e193ce8304ac649aa663e9c982bf9a1a59ecd9d0 WatchSource:0}: Error finding container 055042e5b739245399f2a9d0e193ce8304ac649aa663e9c982bf9a1a59ecd9d0: Status 404 returned error can't find the container with id 055042e5b739245399f2a9d0e193ce8304ac649aa663e9c982bf9a1a59ecd9d0
Sep 17 10:53:27.838378 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:53:27.838343    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d1fe18b_e374_4453_a560_3aa3245d100b.slice/crio-185157e93319d742cd3406307c7251b710cc22dab69d6dd53d77a592ffa107fb WatchSource:0}: Error finding container 185157e93319d742cd3406307c7251b710cc22dab69d6dd53d77a592ffa107fb: Status 404 returned error can't find the container with id 185157e93319d742cd3406307c7251b710cc22dab69d6dd53d77a592ffa107fb
Sep 17 10:53:27.862194 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:27.862167    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wngvt\" (UniqueName: \"kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt\") pod \"network-check-target-fnw2s\" (UID: \"82f7e981-33bb-4353-99bc-1500f39cd125\") " pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:27.862297 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.862285    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 10:53:27.862332 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.862302    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 10:53:27.862332 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.862313    2564 projected.go:196] Error preparing data for projected volume kube-api-access-wngvt for pod openshift-network-diagnostics/network-check-target-fnw2s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 10:53:27.862396 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:27.862353    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt podName:82f7e981-33bb-4353-99bc-1500f39cd125 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:28.862340804 +0000 UTC m=+4.178703939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wngvt" (UniqueName: "kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt") pod "network-check-target-fnw2s" (UID: "82f7e981-33bb-4353-99bc-1500f39cd125") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 10:53:28.077696 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.077625    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 10:53:28.098037 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.098003    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 10:48:27 +0000 UTC" deadline="2028-05-01 16:27:12.160493622 +0000 UTC"
Sep 17 10:53:28.098037 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.098033    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14213h33m44.062463225s"
Sep 17 10:53:28.155780 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.155701    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4" event={"ID":"76f4d810-0354-43d9-871d-f9d94b5d8a98","Type":"ContainerStarted","Data":"6649044f13a349a963b321e75928146f908b4713c36ca870ad53b015ca69c917"}
Sep 17 10:53:28.158167 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.158135    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-phs4b" event={"ID":"a02f264e-0ec3-4ea3-b2a8-aca37526f1c3","Type":"ContainerStarted","Data":"1019f59a1f27f6a9047a613c075c150ed8858bf513c874a53b0de3c30b6783be"}
Sep 17 10:53:28.159612 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.159583    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c559v" event={"ID":"00db4623-b291-4c7e-8674-1eb725b6584d","Type":"ContainerStarted","Data":"50f6247d0f7af1907655cb4ecc8dad430804a1d8171a3c8912e003f6f7f0d849"}
Sep 17 10:53:28.162152 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.162127    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-205.ec2.internal" event={"ID":"e413a23ab9533a4b31565bf83d28336b","Type":"ContainerStarted","Data":"08a44b75037a2f093b7a1d1e563e8995afda821fca6c65760191d52f0348e718"}
Sep 17 10:53:28.163375 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.163346    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fdnxp" event={"ID":"3d1fe18b-e374-4453-a560-3aa3245d100b","Type":"ContainerStarted","Data":"185157e93319d742cd3406307c7251b710cc22dab69d6dd53d77a592ffa107fb"}
Sep 17 10:53:28.164779 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.164752    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wsj8z" event={"ID":"5db558b4-e831-4239-9486-ca2420b01bd7","Type":"ContainerStarted","Data":"055042e5b739245399f2a9d0e193ce8304ac649aa663e9c982bf9a1a59ecd9d0"}
Sep 17 10:53:28.165973 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.165948    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-thp9w" event={"ID":"cff16177-6fce-4305-8de0-742e7c49db64","Type":"ContainerStarted","Data":"d9c5b5df05748c345c5c324f359ef1a04bf9e3b0a60d0cc6b78152fdae4c1481"}
Sep 17 10:53:28.167297 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.167261    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d7dvr" event={"ID":"ae000258-0b0b-4b77-a6d7-85091d242316","Type":"ContainerStarted","Data":"febfef32e9b4ea02e74d566b8224d194c208d714542699bbf107a871abc1916c"}
Sep 17 10:53:28.168636 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.168591    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rdz5r" event={"ID":"f7775776-e097-4507-9a6a-5d21c14ebe07","Type":"ContainerStarted","Data":"992715be0aaacc105fa82e5a25b7074a0a8f0340d85ea7acfc640e474a6c3349"}
Sep 17 10:53:28.170008 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.169983    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jnlqq" event={"ID":"d25df7be-fd4f-4a61-8313-ebbe3f18fcd8","Type":"ContainerStarted","Data":"2827156aba400f3fbec6d1308d33b2a1836fcfeb43e1b27e3beea3dbbd6d0712"}
Sep 17 10:53:28.181458 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.181396    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-205.ec2.internal" podStartSLOduration=1.181380709 podStartE2EDuration="1.181380709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 10:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:53:28.181319108 +0000 UTC m=+3.497682267" watchObservedRunningTime="2026-09-17 10:53:28.181380709 +0000 UTC m=+3.497743870"
Sep 17 10:53:28.770553 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.770512    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs\") pod \"network-metrics-daemon-g7wtx\" (UID: \"c0028c99-f81b-4054-9098-152b859dd374\") " pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:28.770722 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.770563    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret\") pod \"global-pull-secret-syncer-ddcnf\" (UID: \"909dc7e0-fcc5-4736-8874-8d5206f3f741\") " pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:28.770722 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:28.770684    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 10:53:28.770823 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:28.770744    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret podName:909dc7e0-fcc5-4736-8874-8d5206f3f741 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:30.770725764 +0000 UTC m=+6.087088915 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret") pod "global-pull-secret-syncer-ddcnf" (UID: "909dc7e0-fcc5-4736-8874-8d5206f3f741") : object "kube-system"/"original-pull-secret" not registered
Sep 17 10:53:28.771249 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:28.771226    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 10:53:28.771384 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:28.771287    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs podName:c0028c99-f81b-4054-9098-152b859dd374 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:30.771270084 +0000 UTC m=+6.087633234 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs") pod "network-metrics-daemon-g7wtx" (UID: "c0028c99-f81b-4054-9098-152b859dd374") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 10:53:28.871391 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:28.871351    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wngvt\" (UniqueName: \"kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt\") pod \"network-check-target-fnw2s\" (UID: \"82f7e981-33bb-4353-99bc-1500f39cd125\") " pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:28.871572 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:28.871521    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 10:53:28.871572 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:28.871544    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 10:53:28.871572 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:28.871557    2564 projected.go:196] Error preparing data for projected volume kube-api-access-wngvt for pod openshift-network-diagnostics/network-check-target-fnw2s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 10:53:28.871750 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:28.871614    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt podName:82f7e981-33bb-4353-99bc-1500f39cd125 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:30.871596488 +0000 UTC m=+6.187959629 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wngvt" (UniqueName: "kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt") pod "network-check-target-fnw2s" (UID: "82f7e981-33bb-4353-99bc-1500f39cd125") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 10:53:29.098904 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:29.098667    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 10:48:27 +0000 UTC" deadline="2028-06-03 10:17:37.105225898 +0000 UTC"
Sep 17 10:53:29.098904 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:29.098733    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14999h24m8.006497439s"
Sep 17 10:53:29.136765 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:29.136734    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:29.136951 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:29.136867    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:29.137382 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:29.137356    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:29.137477 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:29.137460    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:29.137701 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:29.137569    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:29.137701 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:29.137662    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:30.787628 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:30.787303    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs\") pod \"network-metrics-daemon-g7wtx\" (UID: \"c0028c99-f81b-4054-9098-152b859dd374\") " pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:30.787628 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:30.787338    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret\") pod \"global-pull-secret-syncer-ddcnf\" (UID: \"909dc7e0-fcc5-4736-8874-8d5206f3f741\") " pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:30.787628 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:30.787463    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 10:53:30.787628 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:30.787525    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret podName:909dc7e0-fcc5-4736-8874-8d5206f3f741 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:34.78751064 +0000 UTC m=+10.103873775 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret") pod "global-pull-secret-syncer-ddcnf" (UID: "909dc7e0-fcc5-4736-8874-8d5206f3f741") : object "kube-system"/"original-pull-secret" not registered
Sep 17 10:53:30.788302 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:30.787936    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 10:53:30.788302 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:30.788012    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs podName:c0028c99-f81b-4054-9098-152b859dd374 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:34.787994192 +0000 UTC m=+10.104357335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs") pod "network-metrics-daemon-g7wtx" (UID: "c0028c99-f81b-4054-9098-152b859dd374") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 10:53:30.888522 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:30.888485    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wngvt\" (UniqueName: \"kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt\") pod \"network-check-target-fnw2s\" (UID: \"82f7e981-33bb-4353-99bc-1500f39cd125\") " pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:30.888698 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:30.888684    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 10:53:30.888767 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:30.888705    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 10:53:30.888767 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:30.888718    2564 projected.go:196] Error preparing data for projected volume kube-api-access-wngvt for pod openshift-network-diagnostics/network-check-target-fnw2s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 10:53:30.888877 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:30.888778    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt podName:82f7e981-33bb-4353-99bc-1500f39cd125 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:34.888757498 +0000 UTC m=+10.205120649 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wngvt" (UniqueName: "kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt") pod "network-check-target-fnw2s" (UID: "82f7e981-33bb-4353-99bc-1500f39cd125") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 10:53:31.138802 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:31.138708    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:31.138802 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:31.138725    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:31.139043 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:31.138858    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:31.139043 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:31.138863    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:31.139043 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:31.138954    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:31.139043 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:31.139019    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:32.211886 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:32.211607    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal" event={"ID":"bb68f00172f0a347af69ef95526d7c0e","Type":"ContainerStarted","Data":"795a450e1282951bb4d640c0dd843eb5ffa27590c6e66a121549440bf6d8f77d"}
Sep 17 10:53:33.139580 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:33.139547    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:33.139764 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:33.139690    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:33.140156 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:33.140129    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:33.140271 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:33.140170    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:33.140331 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:33.140261    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:33.140398 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:33.140355    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:34.825297 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:34.825261    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs\") pod \"network-metrics-daemon-g7wtx\" (UID: \"c0028c99-f81b-4054-9098-152b859dd374\") " pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:34.825798 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:34.825311    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret\") pod \"global-pull-secret-syncer-ddcnf\" (UID: \"909dc7e0-fcc5-4736-8874-8d5206f3f741\") " pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:34.825798 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:34.825461    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 10:53:34.825798 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:34.825516    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret podName:909dc7e0-fcc5-4736-8874-8d5206f3f741 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:42.825502684 +0000 UTC m=+18.141865817 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret") pod "global-pull-secret-syncer-ddcnf" (UID: "909dc7e0-fcc5-4736-8874-8d5206f3f741") : object "kube-system"/"original-pull-secret" not registered
Sep 17 10:53:34.825987 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:34.825848    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 10:53:34.825987 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:34.825913    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs podName:c0028c99-f81b-4054-9098-152b859dd374 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:42.825881198 +0000 UTC m=+18.142244331 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs") pod "network-metrics-daemon-g7wtx" (UID: "c0028c99-f81b-4054-9098-152b859dd374") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 10:53:34.926201 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:34.926162    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wngvt\" (UniqueName: \"kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt\") pod \"network-check-target-fnw2s\" (UID: \"82f7e981-33bb-4353-99bc-1500f39cd125\") " pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:34.926370 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:34.926313    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 10:53:34.926370 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:34.926334    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 10:53:34.926370 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:34.926348    2564 projected.go:196] Error preparing data for projected volume kube-api-access-wngvt for pod openshift-network-diagnostics/network-check-target-fnw2s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 10:53:34.926498 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:34.926411    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt podName:82f7e981-33bb-4353-99bc-1500f39cd125 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:42.926394263 +0000 UTC m=+18.242757402 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wngvt" (UniqueName: "kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt") pod "network-check-target-fnw2s" (UID: "82f7e981-33bb-4353-99bc-1500f39cd125") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 10:53:35.137822 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:35.137746    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:35.138014 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:35.137857    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:35.138014 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:35.137937    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:35.138143 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:35.138021    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:35.138143 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:35.138042    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:35.138323 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:35.138186    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:37.138563 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:37.138533    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:37.138970 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:37.138639    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:37.139027 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:37.139016    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:37.139123 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:37.139103    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:37.139171 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:37.139162    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:37.139245 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:37.139231    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:37.225067 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:37.225034    2564 generic.go:356] "Generic (PLEG): container finished" podID="bb68f00172f0a347af69ef95526d7c0e" containerID="795a450e1282951bb4d640c0dd843eb5ffa27590c6e66a121549440bf6d8f77d" exitCode=0
Sep 17 10:53:37.225228 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:37.225106    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal" event={"ID":"bb68f00172f0a347af69ef95526d7c0e","Type":"ContainerDied","Data":"795a450e1282951bb4d640c0dd843eb5ffa27590c6e66a121549440bf6d8f77d"}
Sep 17 10:53:39.137098 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:39.137059    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:39.137538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:39.137066    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:39.137538 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:39.137201    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:39.137538 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:39.137204    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:39.137538 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:39.137317    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:39.137538 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:39.137394    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:41.137115 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:41.137085    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:41.137584 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:41.137085    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:41.137584 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:41.137243    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:41.137584 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:41.137266    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:41.137584 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:41.137391    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:41.138008 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:41.137966    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:42.886936 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:42.886884    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs\") pod \"network-metrics-daemon-g7wtx\" (UID: \"c0028c99-f81b-4054-9098-152b859dd374\") " pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:42.886936 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:42.886941    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret\") pod \"global-pull-secret-syncer-ddcnf\" (UID: \"909dc7e0-fcc5-4736-8874-8d5206f3f741\") " pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:42.887490 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:42.887069    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 10:53:42.887490 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:42.887078    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 10:53:42.887490 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:42.887140    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs podName:c0028c99-f81b-4054-9098-152b859dd374 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:58.887119946 +0000 UTC m=+34.203483079 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs") pod "network-metrics-daemon-g7wtx" (UID: "c0028c99-f81b-4054-9098-152b859dd374") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 10:53:42.887490 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:42.887164    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret podName:909dc7e0-fcc5-4736-8874-8d5206f3f741 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:58.887153376 +0000 UTC m=+34.203516528 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret") pod "global-pull-secret-syncer-ddcnf" (UID: "909dc7e0-fcc5-4736-8874-8d5206f3f741") : object "kube-system"/"original-pull-secret" not registered
Sep 17 10:53:42.987597 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:42.987564    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wngvt\" (UniqueName: \"kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt\") pod \"network-check-target-fnw2s\" (UID: \"82f7e981-33bb-4353-99bc-1500f39cd125\") " pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:42.987823 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:42.987803    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 10:53:42.987861 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:42.987836    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 10:53:42.987861 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:42.987850    2564 projected.go:196] Error preparing data for projected volume kube-api-access-wngvt for pod openshift-network-diagnostics/network-check-target-fnw2s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 10:53:42.987958 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:42.987927    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt podName:82f7e981-33bb-4353-99bc-1500f39cd125 nodeName:}" failed. No retries permitted until 2026-09-17 10:53:58.987909296 +0000 UTC m=+34.304272445 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wngvt" (UniqueName: "kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt") pod "network-check-target-fnw2s" (UID: "82f7e981-33bb-4353-99bc-1500f39cd125") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 10:53:43.137290 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:43.137213    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:43.137430 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:43.137216    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:43.137487 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:43.137467    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:43.137536 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:43.137357    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:43.137536 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:43.137513    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:43.137603 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:43.137576    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:45.137679 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:45.137646    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:45.138105 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:45.137759    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:45.138105 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:45.137842    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:45.138105 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:45.137955    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:45.138105 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:45.138000    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:45.138105 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:45.138071    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:47.136849 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.136677    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:47.137620 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.136677    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:47.137620 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:47.136944    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:47.137620 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.136677    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:47.137620 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:47.137020    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:47.137620 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:47.137083    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:47.243317 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.243291    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rdz5r" event={"ID":"f7775776-e097-4507-9a6a-5d21c14ebe07","Type":"ContainerStarted","Data":"3be7e74e7be936ad35c37f820385e84575bfed4adfccdf2784f5e57fa73b5633"}
Sep 17 10:53:47.245148 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.245126    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal" event={"ID":"bb68f00172f0a347af69ef95526d7c0e","Type":"ContainerStarted","Data":"76d35c6a915ce865cef4d766cede92ffc9e890c6aefdeae1be2ae95399c03c18"}
Sep 17 10:53:47.247647 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.247625    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4" event={"ID":"76f4d810-0354-43d9-871d-f9d94b5d8a98","Type":"ContainerStarted","Data":"d632aa858d73b2ad5942c20a8a9478ee1aa7ac2622f03fd11312a00d5680d415"}
Sep 17 10:53:47.249479 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.249460    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-phs4b" event={"ID":"a02f264e-0ec3-4ea3-b2a8-aca37526f1c3","Type":"ContainerStarted","Data":"1a3d48c4e41f70f66f2b49858cca5619a3ae02923d1a4f0340e3bfd489e1671c"}
Sep 17 10:53:47.252097 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.252077    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c559v" event={"ID":"00db4623-b291-4c7e-8674-1eb725b6584d","Type":"ContainerStarted","Data":"afdfc79d2d6e30646cc39596317179bc70648081f05abb3f2f91d78af232141d"}
Sep 17 10:53:47.252188 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.252137    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c559v" event={"ID":"00db4623-b291-4c7e-8674-1eb725b6584d","Type":"ContainerStarted","Data":"b305336a9cd49d460e278b0f432e60cb5b5fd804a2527acb4bcef732a14f7740"}
Sep 17 10:53:47.252188 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.252174    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c559v" event={"ID":"00db4623-b291-4c7e-8674-1eb725b6584d","Type":"ContainerStarted","Data":"3f67b97a0e90454b8b3eaeeb725d6a8c1db574ec6b0ee460f0aee6b5069695e1"}
Sep 17 10:53:47.252291 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.252209    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c559v" event={"ID":"00db4623-b291-4c7e-8674-1eb725b6584d","Type":"ContainerStarted","Data":"2d05cf4b5741777162fdb20d6eb9114b387a22295b1ee9b7ef24924f8a2f9892"}
Sep 17 10:53:47.252291 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.252238    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c559v" event={"ID":"00db4623-b291-4c7e-8674-1eb725b6584d","Type":"ContainerStarted","Data":"42e23f1a0092eef6214ba16f3671f0040e630a5c1f2a5715d86104c3a5c4d3e3"}
Sep 17 10:53:47.253418 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.253399    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fdnxp" event={"ID":"3d1fe18b-e374-4453-a560-3aa3245d100b","Type":"ContainerStarted","Data":"4006ae8bf7b03e998723df622b3702b29edf8b3c87bbf836f99b3c26dac0b1ba"}
Sep 17 10:53:47.254883 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.254862    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wsj8z" event={"ID":"5db558b4-e831-4239-9486-ca2420b01bd7","Type":"ContainerStarted","Data":"e1d84bc8f21b490880a49b0e9a8937e061e9bca1b9fdef08de02772dd50383f1"}
Sep 17 10:53:47.256262 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.256239    2564 generic.go:356] "Generic (PLEG): container finished" podID="cff16177-6fce-4305-8de0-742e7c49db64" containerID="d72a5a6444f400a082f775e881ea8d24186ab16221a4c4736a1d7ce6bd4c82aa" exitCode=0
Sep 17 10:53:47.256336 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.256310    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-thp9w" event={"ID":"cff16177-6fce-4305-8de0-742e7c49db64","Type":"ContainerDied","Data":"d72a5a6444f400a082f775e881ea8d24186ab16221a4c4736a1d7ce6bd4c82aa"}
Sep 17 10:53:47.257969 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.257949    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d7dvr" event={"ID":"ae000258-0b0b-4b77-a6d7-85091d242316","Type":"ContainerStarted","Data":"bc8f2bfa304a924d04231d4cbcd7829cb9a906eb7a27d0c2c9c6b4ab2379ecc4"}
Sep 17 10:53:47.299858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.299819    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rdz5r" podStartSLOduration=2.511730435 podStartE2EDuration="21.299804401s" totalImagesPullingTime="18.788073966s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 10:53:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:53:47.265294553 +0000 UTC m=+22.581657709" watchObservedRunningTime="2026-09-17 10:53:47.299804401 +0000 UTC m=+22.616167555"
Sep 17 10:53:47.336812 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.336642    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-phs4b" podStartSLOduration=3.546747194 podStartE2EDuration="22.33662803s" totalImagesPullingTime="18.789880836s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 10:53:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:53:47.317273305 +0000 UTC m=+22.633636471" watchObservedRunningTime="2026-09-17 10:53:47.33662803 +0000 UTC m=+22.652991185"
Sep 17 10:53:47.337200 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.337175    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wsj8z" podStartSLOduration=3.527041936 podStartE2EDuration="22.337168599s" totalImagesPullingTime="18.810126663s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 10:53:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:53:47.336558834 +0000 UTC m=+22.652922002" watchObservedRunningTime="2026-09-17 10:53:47.337168599 +0000 UTC m=+22.653531754"
Sep 17 10:53:47.356216 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.356159    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fdnxp" podStartSLOduration=2.603450369 podStartE2EDuration="21.356147921s" totalImagesPullingTime="18.752697552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:53:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:53:47.355739438 +0000 UTC m=+22.672102594" watchObservedRunningTime="2026-09-17 10:53:47.356147921 +0000 UTC m=+22.672511073"
Sep 17 10:53:47.406867 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.406828    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-d7dvr" podStartSLOduration=3.625232418 podStartE2EDuration="22.406816553s" totalImagesPullingTime="18.781584135s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 10:53:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:53:47.382559357 +0000 UTC m=+22.698922512" watchObservedRunningTime="2026-09-17 10:53:47.406816553 +0000 UTC m=+22.723179705"
Sep 17 10:53:47.407019 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:47.406959    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-205.ec2.internal" podStartSLOduration=20.40695408 podStartE2EDuration="20.40695408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 10:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:53:47.406666719 +0000 UTC m=+22.723029885" watchObservedRunningTime="2026-09-17 10:53:47.40695408 +0000 UTC m=+22.723317234"
Sep 17 10:53:48.274392 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:48.274361    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c559v" event={"ID":"00db4623-b291-4c7e-8674-1eb725b6584d","Type":"ContainerStarted","Data":"0eb8c1bc869dc9ff7623cc4946d866aa1ce0607df4c67ff91cd82a3bfc2e4b4b"}
Sep 17 10:53:48.276015 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:48.275992    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jnlqq" event={"ID":"d25df7be-fd4f-4a61-8313-ebbe3f18fcd8","Type":"ContainerStarted","Data":"6933438e5a6c398d3e2063192399828b16e4fb8c7f847d99a47e0b447c27fdb4"}
Sep 17 10:53:48.293038 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:48.293002    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jnlqq" podStartSLOduration=3.505762447 podStartE2EDuration="22.292990222s" totalImagesPullingTime="18.787227775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 10:53:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:53:48.292910393 +0000 UTC m=+23.609273541" watchObservedRunningTime="2026-09-17 10:53:48.292990222 +0000 UTC m=+23.609353377"
Sep 17 10:53:48.349673 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:48.349503    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 10:53:49.109748 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:49.109626    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T10:53:48.349514386Z","UUID":"877778ac-3c02-47f7-9c3e-e435a3b6faf9","Handler":null,"Name":"","Endpoint":""}
Sep 17 10:53:49.111466 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:49.111444    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 10:53:49.111466 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:49.111472    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 10:53:49.136850 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:49.136823    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:49.137006 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:49.136828    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:49.137006 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:49.136955    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:49.137006 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:49.136955    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:49.137166 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:49.137061    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:49.137166 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:49.137138    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:49.280186 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:49.280153    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4" event={"ID":"76f4d810-0354-43d9-871d-f9d94b5d8a98","Type":"ContainerStarted","Data":"3fe47e5d4d7776a84fe439a588364244677c64823acb6f23f525b70f686bee77"}
Sep 17 10:53:49.828616 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:49.828580    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fdnxp"
Sep 17 10:53:49.829380 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:49.829360    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fdnxp"
Sep 17 10:53:50.285386 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:50.285350    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4" event={"ID":"76f4d810-0354-43d9-871d-f9d94b5d8a98","Type":"ContainerStarted","Data":"24a1a07249e033233bb213ce2a8cfe753b776cd5ab3c9265f4a4359a9ea123f4"}
Sep 17 10:53:50.290297 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:50.290267    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c559v" event={"ID":"00db4623-b291-4c7e-8674-1eb725b6584d","Type":"ContainerStarted","Data":"e8a7cdeeb52a359b678f116415282e62ca3687e1bd5e23f64b2f0c388ed77fcf"}
Sep 17 10:53:50.308944 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:50.308876    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpk4" podStartSLOduration=3.082924314 podStartE2EDuration="24.308864031s" totalImagesPullingTime="21.225939717s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 10:53:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:53:50.3081584 +0000 UTC m=+25.624521556" watchObservedRunningTime="2026-09-17 10:53:50.308864031 +0000 UTC m=+25.625227185"
Sep 17 10:53:51.137247 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:51.137215    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:51.137449 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:51.137217    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:51.137449 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:51.137339    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:51.137449 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:51.137348    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:51.137612 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:51.137450    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:51.137612 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:51.137530    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:53.136859 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:53.136603    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:53.137507 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:53.136603    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:53.137507 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:53.136636    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:53.137507 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:53.136992    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:53.137507 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:53.137092    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:53.137507 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:53.137185    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:53.153635 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:53.153612    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fdnxp"
Sep 17 10:53:53.153766 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:53.153695    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 17 10:53:53.154199 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:53.154177    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fdnxp"
Sep 17 10:53:53.300416 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:53.300391    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c559v" event={"ID":"00db4623-b291-4c7e-8674-1eb725b6584d","Type":"ContainerStarted","Data":"4716bf204e7e658903dc8a585210557a503297f97f86b1b3653d00c8371f17c8"}
Sep 17 10:53:53.300613 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:53.300597    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:53.302355 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:53.302335    2564 generic.go:356] "Generic (PLEG): container finished" podID="cff16177-6fce-4305-8de0-742e7c49db64" containerID="aca1dcac0dc1c5d6ea9c7297e760f2783d5679b19e16edef3106c5c5b7d7d9f0" exitCode=0
Sep 17 10:53:53.302435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:53.302406    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-thp9w" event={"ID":"cff16177-6fce-4305-8de0-742e7c49db64","Type":"ContainerDied","Data":"aca1dcac0dc1c5d6ea9c7297e760f2783d5679b19e16edef3106c5c5b7d7d9f0"}
Sep 17 10:53:53.316561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:53.316540    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:53.369264 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:53.369224    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-c559v" podStartSLOduration=8.527444469 podStartE2EDuration="27.369212269s" totalImagesPullingTime="18.8417678s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 10:53:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:53:53.340588691 +0000 UTC m=+28.656951843" watchObservedRunningTime="2026-09-17 10:53:53.369212269 +0000 UTC m=+28.685575457"
Sep 17 10:53:54.305217 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:54.304996    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:54.305693 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:54.305318    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:54.321180 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:54.321160    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:53:54.450117 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:54.450093    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g7wtx"]
Sep 17 10:53:54.450247 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:54.450224    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:54.450379 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:54.450350    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:54.452503 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:54.452479    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ddcnf"]
Sep 17 10:53:54.452612 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:54.452582    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:54.452701 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:54.452681    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:54.453794 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:54.453732    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fnw2s"]
Sep 17 10:53:54.453889 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:54.453820    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:54.453965 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:54.453919    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:55.308105 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:55.308077    2564 generic.go:356] "Generic (PLEG): container finished" podID="cff16177-6fce-4305-8de0-742e7c49db64" containerID="785fd407b71d5cf6f1fc6a6251b7058813b1727823915596e14b9d974294d659" exitCode=0
Sep 17 10:53:55.308420 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:55.308155    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-thp9w" event={"ID":"cff16177-6fce-4305-8de0-742e7c49db64","Type":"ContainerDied","Data":"785fd407b71d5cf6f1fc6a6251b7058813b1727823915596e14b9d974294d659"}
Sep 17 10:53:56.136443 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:56.136412    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:56.136443 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:56.136431    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:56.136654 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:56.136524    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:56.136654 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:56.136527    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:56.136654 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:56.136591    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:56.136748 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:56.136672    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:57.314963 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:57.314930    2564 generic.go:356] "Generic (PLEG): container finished" podID="cff16177-6fce-4305-8de0-742e7c49db64" containerID="5463051bbcfabbc60a121fd03b550b044a2478269db368c7077a1d83f91cf1f1" exitCode=0
Sep 17 10:53:57.314963 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:57.314969    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-thp9w" event={"ID":"cff16177-6fce-4305-8de0-742e7c49db64","Type":"ContainerDied","Data":"5463051bbcfabbc60a121fd03b550b044a2478269db368c7077a1d83f91cf1f1"}
Sep 17 10:53:58.136800 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:58.136763    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:58.137013 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:58.136813    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:58.137013 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:58.136763    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:58.137013 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:58.136941    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wtx" podUID="c0028c99-f81b-4054-9098-152b859dd374"
Sep 17 10:53:58.137197 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:58.137040    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fnw2s" podUID="82f7e981-33bb-4353-99bc-1500f39cd125"
Sep 17 10:53:58.137197 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:58.137151    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ddcnf" podUID="909dc7e0-fcc5-4736-8874-8d5206f3f741"
Sep 17 10:53:58.900530 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:58.900494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs\") pod \"network-metrics-daemon-g7wtx\" (UID: \"c0028c99-f81b-4054-9098-152b859dd374\") " pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:53:58.901034 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:58.900544    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret\") pod \"global-pull-secret-syncer-ddcnf\" (UID: \"909dc7e0-fcc5-4736-8874-8d5206f3f741\") " pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:53:58.901034 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:58.900622    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 10:53:58.901034 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:58.900658    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 10:53:58.901034 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:58.900699    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs podName:c0028c99-f81b-4054-9098-152b859dd374 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:30.900677284 +0000 UTC m=+66.217040432 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs") pod "network-metrics-daemon-g7wtx" (UID: "c0028c99-f81b-4054-9098-152b859dd374") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 10:53:58.901034 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:58.900719    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret podName:909dc7e0-fcc5-4736-8874-8d5206f3f741 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:30.900708911 +0000 UTC m=+66.217072050 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret") pod "global-pull-secret-syncer-ddcnf" (UID: "909dc7e0-fcc5-4736-8874-8d5206f3f741") : object "kube-system"/"original-pull-secret" not registered
Sep 17 10:53:59.001312 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.001279    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wngvt\" (UniqueName: \"kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt\") pod \"network-check-target-fnw2s\" (UID: \"82f7e981-33bb-4353-99bc-1500f39cd125\") " pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:53:59.001469 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:59.001407    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 10:53:59.001469 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:59.001425    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 10:53:59.001469 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:59.001437    2564 projected.go:196] Error preparing data for projected volume kube-api-access-wngvt for pod openshift-network-diagnostics/network-check-target-fnw2s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 10:53:59.001625 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:59.001497    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt podName:82f7e981-33bb-4353-99bc-1500f39cd125 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:31.00147858 +0000 UTC m=+66.317841723 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wngvt" (UniqueName: "kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt") pod "network-check-target-fnw2s" (UID: "82f7e981-33bb-4353-99bc-1500f39cd125") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 10:53:59.464007 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.463977    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-205.ec2.internal" event="NodeReady"
Sep 17 10:53:59.464183 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.464106    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 10:53:59.510524 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.510492    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7fb85b79c5-rlhvv"]
Sep 17 10:53:59.531531 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.531365    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-f466l"]
Sep 17 10:53:59.532970 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.531888    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.537213 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.537188    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 17 10:53:59.537609 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.537269    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n4lnv\""
Sep 17 10:53:59.537723 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.537705    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 17 10:53:59.538400 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.538379    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 17 10:53:59.544801 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.544775    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 17 10:53:59.547061 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.546815    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vrd77"]
Sep 17 10:53:59.547352 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.546891    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f466l"
Sep 17 10:53:59.549677 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.549658    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 10:53:59.550014 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.549999    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 10:53:59.550103 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.550035    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lfdp8\""
Sep 17 10:53:59.563101 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.563074    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fb85b79c5-rlhvv"]
Sep 17 10:53:59.563207 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.563113    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vrd77"
Sep 17 10:53:59.563207 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.563124    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f466l"]
Sep 17 10:53:59.563207 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.563151    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vrd77"]
Sep 17 10:53:59.565166 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.565148    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 10:53:59.565297 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.565184    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 10:53:59.565297 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.565148    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 10:53:59.565437 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.565421    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bbfck\""
Sep 17 10:53:59.706938 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.706893    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3d4e645b-517d-42e1-92d2-107039f0fce9-installation-pull-secrets\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.707083 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.706954    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhf67\" (UniqueName: \"kubernetes.io/projected/80044c26-9705-435f-a719-4a0e57ac5c75-kube-api-access-lhf67\") pod \"ingress-canary-vrd77\" (UID: \"80044c26-9705-435f-a719-4a0e57ac5c75\") " pod="openshift-ingress-canary/ingress-canary-vrd77"
Sep 17 10:53:59.707083 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.707008    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80044c26-9705-435f-a719-4a0e57ac5c75-cert\") pod \"ingress-canary-vrd77\" (UID: \"80044c26-9705-435f-a719-4a0e57ac5c75\") " pod="openshift-ingress-canary/ingress-canary-vrd77"
Sep 17 10:53:59.707083 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.707037    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wlbb\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-kube-api-access-5wlbb\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.707198 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.707095    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.707198 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.707117    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-bound-sa-token\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.707198 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.707140    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5cfp\" (UniqueName: \"kubernetes.io/projected/a3b2ce07-6399-4354-8cdd-be6af04d46a8-kube-api-access-f5cfp\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:53:59.707198 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.707189    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:53:59.707347 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.707226    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-certificates\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.707347 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.707247    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3d4e645b-517d-42e1-92d2-107039f0fce9-image-registry-private-configuration\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.707347 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.707266    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3b2ce07-6399-4354-8cdd-be6af04d46a8-tmp-dir\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:53:59.707450 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.707335    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3d4e645b-517d-42e1-92d2-107039f0fce9-ca-trust-extracted\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.707450 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.707379    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3b2ce07-6399-4354-8cdd-be6af04d46a8-config-volume\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:53:59.707450 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.707403    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3d4e645b-517d-42e1-92d2-107039f0fce9-trusted-ca\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.807919 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.807834    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3d4e645b-517d-42e1-92d2-107039f0fce9-installation-pull-secrets\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.807919 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.807867    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhf67\" (UniqueName: \"kubernetes.io/projected/80044c26-9705-435f-a719-4a0e57ac5c75-kube-api-access-lhf67\") pod \"ingress-canary-vrd77\" (UID: \"80044c26-9705-435f-a719-4a0e57ac5c75\") " pod="openshift-ingress-canary/ingress-canary-vrd77"
Sep 17 10:53:59.808203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.808183    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80044c26-9705-435f-a719-4a0e57ac5c75-cert\") pod \"ingress-canary-vrd77\" (UID: \"80044c26-9705-435f-a719-4a0e57ac5c75\") " pod="openshift-ingress-canary/ingress-canary-vrd77"
Sep 17 10:53:59.808286 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.808241    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wlbb\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-kube-api-access-5wlbb\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.808344 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.808288    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.808344 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.808314    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-bound-sa-token\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.808445 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:59.808408    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 10:53:59.808445 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:59.808424    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fb85b79c5-rlhvv: secret "image-registry-tls" not found
Sep 17 10:53:59.808542 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:59.808477    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls podName:3d4e645b-517d-42e1-92d2-107039f0fce9 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:00.308458695 +0000 UTC m=+35.624821845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls") pod "image-registry-7fb85b79c5-rlhvv" (UID: "3d4e645b-517d-42e1-92d2-107039f0fce9") : secret "image-registry-tls" not found
Sep 17 10:53:59.808542 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.808500    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5cfp\" (UniqueName: \"kubernetes.io/projected/a3b2ce07-6399-4354-8cdd-be6af04d46a8-kube-api-access-f5cfp\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:53:59.808542 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.808537    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:53:59.808693 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.808577    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-certificates\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.808693 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.808610    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3d4e645b-517d-42e1-92d2-107039f0fce9-image-registry-private-configuration\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.808693 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.808638    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3b2ce07-6399-4354-8cdd-be6af04d46a8-tmp-dir\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:53:59.808693 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:59.808650    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 10:53:59.808693 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.808663    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3d4e645b-517d-42e1-92d2-107039f0fce9-ca-trust-extracted\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.808954 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.808698    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3b2ce07-6399-4354-8cdd-be6af04d46a8-config-volume\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:53:59.808954 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:53:59.808721    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls podName:a3b2ce07-6399-4354-8cdd-be6af04d46a8 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:00.308702246 +0000 UTC m=+35.625065380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls") pod "dns-default-f466l" (UID: "a3b2ce07-6399-4354-8cdd-be6af04d46a8") : secret "dns-default-metrics-tls" not found
Sep 17 10:53:59.808954 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.808755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3d4e645b-517d-42e1-92d2-107039f0fce9-trusted-ca\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.809300 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.809277    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3d4e645b-517d-42e1-92d2-107039f0fce9-ca-trust-extracted\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.809444 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.809402    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-certificates\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.809705 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.809679    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3b2ce07-6399-4354-8cdd-be6af04d46a8-config-volume\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:53:59.809858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.809681    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3d4e645b-517d-42e1-92d2-107039f0fce9-trusted-ca\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.809858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.809832    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3b2ce07-6399-4354-8cdd-be6af04d46a8-tmp-dir\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:53:59.813279 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.813254    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80044c26-9705-435f-a719-4a0e57ac5c75-cert\") pod \"ingress-canary-vrd77\" (UID: \"80044c26-9705-435f-a719-4a0e57ac5c75\") " pod="openshift-ingress-canary/ingress-canary-vrd77"
Sep 17 10:53:59.813279 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.813268    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3d4e645b-517d-42e1-92d2-107039f0fce9-image-registry-private-configuration\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.813429 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.813268    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3d4e645b-517d-42e1-92d2-107039f0fce9-installation-pull-secrets\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.822491 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.822448    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhf67\" (UniqueName: \"kubernetes.io/projected/80044c26-9705-435f-a719-4a0e57ac5c75-kube-api-access-lhf67\") pod \"ingress-canary-vrd77\" (UID: \"80044c26-9705-435f-a719-4a0e57ac5c75\") " pod="openshift-ingress-canary/ingress-canary-vrd77"
Sep 17 10:53:59.822657 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.822639    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wlbb\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-kube-api-access-5wlbb\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.822740 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.822718    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5cfp\" (UniqueName: \"kubernetes.io/projected/a3b2ce07-6399-4354-8cdd-be6af04d46a8-kube-api-access-f5cfp\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:53:59.823049 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.823029    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-bound-sa-token\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:53:59.873467 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:53:59.873449    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vrd77"
Sep 17 10:54:00.136916 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:00.136823    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:54:00.136916 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:00.136869    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:54:00.137667 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:00.136952    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:54:00.139635 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:00.139612    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 10:54:00.139781 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:00.139665    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4chbd\""
Sep 17 10:54:00.139781 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:00.139698    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 10:54:00.139781 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:00.139618    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gx69l\""
Sep 17 10:54:00.139781 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:00.139715    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 10:54:00.139781 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:00.139612    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 10:54:00.212410 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:00.212383    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vrd77"]
Sep 17 10:54:00.216272 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:54:00.216248    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80044c26_9705_435f_a719_4a0e57ac5c75.slice/crio-d74d6fda42e011399bb7c0e2cb2e790b3bae1c7dc16fce3c3a5cbab8d076d379 WatchSource:0}: Error finding container d74d6fda42e011399bb7c0e2cb2e790b3bae1c7dc16fce3c3a5cbab8d076d379: Status 404 returned error can't find the container with id d74d6fda42e011399bb7c0e2cb2e790b3bae1c7dc16fce3c3a5cbab8d076d379
Sep 17 10:54:00.313556 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:00.313528    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:54:00.313556 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:00.313560    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:54:00.313732 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:00.313656    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 10:54:00.313732 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:00.313666    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fb85b79c5-rlhvv: secret "image-registry-tls" not found
Sep 17 10:54:00.313732 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:00.313703    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls podName:3d4e645b-517d-42e1-92d2-107039f0fce9 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:01.313691065 +0000 UTC m=+36.630054198 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls") pod "image-registry-7fb85b79c5-rlhvv" (UID: "3d4e645b-517d-42e1-92d2-107039f0fce9") : secret "image-registry-tls" not found
Sep 17 10:54:00.313930 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:00.313731    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 10:54:00.313930 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:00.313775    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls podName:a3b2ce07-6399-4354-8cdd-be6af04d46a8 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:01.313763422 +0000 UTC m=+36.630126587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls") pod "dns-default-f466l" (UID: "a3b2ce07-6399-4354-8cdd-be6af04d46a8") : secret "dns-default-metrics-tls" not found
Sep 17 10:54:00.322999 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:00.322977    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vrd77" event={"ID":"80044c26-9705-435f-a719-4a0e57ac5c75","Type":"ContainerStarted","Data":"d74d6fda42e011399bb7c0e2cb2e790b3bae1c7dc16fce3c3a5cbab8d076d379"}
Sep 17 10:54:01.322136 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:01.322090    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:54:01.322721 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:01.322151    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:54:01.322721 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:01.322214    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 10:54:01.322721 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:01.322239    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fb85b79c5-rlhvv: secret "image-registry-tls" not found
Sep 17 10:54:01.322721 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:01.322297    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 10:54:01.322721 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:01.322304    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls podName:3d4e645b-517d-42e1-92d2-107039f0fce9 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:03.322286673 +0000 UTC m=+38.638649810 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls") pod "image-registry-7fb85b79c5-rlhvv" (UID: "3d4e645b-517d-42e1-92d2-107039f0fce9") : secret "image-registry-tls" not found
Sep 17 10:54:01.322721 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:01.322386    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls podName:a3b2ce07-6399-4354-8cdd-be6af04d46a8 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:03.322372095 +0000 UTC m=+38.638735232 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls") pod "dns-default-f466l" (UID: "a3b2ce07-6399-4354-8cdd-be6af04d46a8") : secret "dns-default-metrics-tls" not found
Sep 17 10:54:02.333924 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:02.332839    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-thp9w" event={"ID":"cff16177-6fce-4305-8de0-742e7c49db64","Type":"ContainerStarted","Data":"27af3ce2d5a256c5af54c357f392b632caaf08ae2759de1a22a16344c0e189ef"}
Sep 17 10:54:03.338779 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:03.338590    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:54:03.339189 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:03.338804    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:54:03.339189 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:03.338691    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 10:54:03.339189 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:03.338851    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fb85b79c5-rlhvv: secret "image-registry-tls" not found
Sep 17 10:54:03.339189 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:03.338930    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls podName:3d4e645b-517d-42e1-92d2-107039f0fce9 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:07.338889404 +0000 UTC m=+42.655252537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls") pod "image-registry-7fb85b79c5-rlhvv" (UID: "3d4e645b-517d-42e1-92d2-107039f0fce9") : secret "image-registry-tls" not found
Sep 17 10:54:03.339189 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:03.338944    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 10:54:03.339189 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:03.338975    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls podName:a3b2ce07-6399-4354-8cdd-be6af04d46a8 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:07.338965689 +0000 UTC m=+42.655328836 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls") pod "dns-default-f466l" (UID: "a3b2ce07-6399-4354-8cdd-be6af04d46a8") : secret "dns-default-metrics-tls" not found
Sep 17 10:54:03.339189 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:03.339043    2564 generic.go:356] "Generic (PLEG): container finished" podID="cff16177-6fce-4305-8de0-742e7c49db64" containerID="27af3ce2d5a256c5af54c357f392b632caaf08ae2759de1a22a16344c0e189ef" exitCode=0
Sep 17 10:54:03.339189 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:03.339109    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-thp9w" event={"ID":"cff16177-6fce-4305-8de0-742e7c49db64","Type":"ContainerDied","Data":"27af3ce2d5a256c5af54c357f392b632caaf08ae2759de1a22a16344c0e189ef"}
Sep 17 10:54:04.344827 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:04.344799    2564 generic.go:356] "Generic (PLEG): container finished" podID="cff16177-6fce-4305-8de0-742e7c49db64" containerID="3d865adabbef167e0eeef6be3a99c02d15a22822b52e942b4f56642b8e20e27a" exitCode=0
Sep 17 10:54:04.345165 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:04.344836    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-thp9w" event={"ID":"cff16177-6fce-4305-8de0-742e7c49db64","Type":"ContainerDied","Data":"3d865adabbef167e0eeef6be3a99c02d15a22822b52e942b4f56642b8e20e27a"}
Sep 17 10:54:05.349078 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:05.349046    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vrd77" event={"ID":"80044c26-9705-435f-a719-4a0e57ac5c75","Type":"ContainerStarted","Data":"20e215ca6eb59d81df37541a19f9941c5cf85dbbcbf5c59df0a40657ce7b4c41"}
Sep 17 10:54:05.352425 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:05.352401    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-thp9w" event={"ID":"cff16177-6fce-4305-8de0-742e7c49db64","Type":"ContainerStarted","Data":"ba8768d8c23238fd966e4f279f3d3e2b141c8f4291fd769d97fc98df221f79d3"}
Sep 17 10:54:05.368223 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:05.368181    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vrd77" podStartSLOduration=2.297490438 podStartE2EDuration="6.368165583s" totalImagesPullingTime="4.070675145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:54:05.367680246 +0000 UTC m=+40.684043401" watchObservedRunningTime="2026-09-17 10:54:05.368165583 +0000 UTC m=+40.684528735"
Sep 17 10:54:05.390579 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:05.390538    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-thp9w" podStartSLOduration=9.360755233999999 podStartE2EDuration="40.390522019s" totalImagesPullingTime="31.029766785s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 10:53:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:54:05.389403167 +0000 UTC m=+40.705766322" watchObservedRunningTime="2026-09-17 10:54:05.390522019 +0000 UTC m=+40.706885173"
Sep 17 10:54:07.367482 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:07.367447    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:54:07.367825 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:07.367488    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:54:07.367825 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:07.367597    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 10:54:07.367825 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:07.367611    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fb85b79c5-rlhvv: secret "image-registry-tls" not found
Sep 17 10:54:07.367825 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:07.367614    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 10:54:07.367825 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:07.367663    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls podName:3d4e645b-517d-42e1-92d2-107039f0fce9 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:15.367648401 +0000 UTC m=+50.684011533 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls") pod "image-registry-7fb85b79c5-rlhvv" (UID: "3d4e645b-517d-42e1-92d2-107039f0fce9") : secret "image-registry-tls" not found
Sep 17 10:54:07.367825 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:07.367676    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls podName:a3b2ce07-6399-4354-8cdd-be6af04d46a8 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:15.367670641 +0000 UTC m=+50.684033774 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls") pod "dns-default-f466l" (UID: "a3b2ce07-6399-4354-8cdd-be6af04d46a8") : secret "dns-default-metrics-tls" not found
Sep 17 10:54:10.183634 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.183607    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nwtnk"]
Sep 17 10:54:10.217576 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.217546    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nwtnk"]
Sep 17 10:54:10.217727 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.217666    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:10.224368 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.224349    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 10:54:10.224484 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.224373    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 10:54:10.224484 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.224352    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 10:54:10.224711 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.224693    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 10:54:10.225483 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.225467    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8sqhr\""
Sep 17 10:54:10.386812 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.386785    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:10.386812 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.386816    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-data-volume\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:10.387015 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.386837    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68478\" (UniqueName: \"kubernetes.io/projected/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-api-access-68478\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:10.387015 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.386863    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-crio-socket\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:10.387015 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.386935    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:10.488264 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.488234    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:10.488335 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.488270    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-data-volume\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:10.488335 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.488286    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68478\" (UniqueName: \"kubernetes.io/projected/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-api-access-68478\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:10.488335 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.488304    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-crio-socket\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:10.488335 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.488328    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:10.488481 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:10.488361    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 10:54:10.488481 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:10.488440    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-rbac-proxy-cm podName:79ee78ea-aacc-4ecb-a534-f4a8cfd4e349 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:10.988425353 +0000 UTC m=+46.304788487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nwtnk" (UID: "79ee78ea-aacc-4ecb-a534-f4a8cfd4e349") : configmap "kube-rbac-proxy" not found
Sep 17 10:54:10.488481 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:10.488447    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 10:54:10.488607 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:10.488504    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls podName:79ee78ea-aacc-4ecb-a534-f4a8cfd4e349 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:10.988488889 +0000 UTC m=+46.304852022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nwtnk" (UID: "79ee78ea-aacc-4ecb-a534-f4a8cfd4e349") : secret "insights-runtime-extractor-tls" not found
Sep 17 10:54:10.488607 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.488529    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-crio-socket\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:10.488682 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.488633    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-data-volume\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:10.505013 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.504983    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68478\" (UniqueName: \"kubernetes.io/projected/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-api-access-68478\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:10.992360 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.992328    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:10.992514 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:10.992455    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:10.992514 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:10.992483    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 10:54:10.992586 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:10.992524    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 10:54:10.992586 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:10.992565    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls podName:79ee78ea-aacc-4ecb-a534-f4a8cfd4e349 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:11.992546469 +0000 UTC m=+47.308909622 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nwtnk" (UID: "79ee78ea-aacc-4ecb-a534-f4a8cfd4e349") : secret "insights-runtime-extractor-tls" not found
Sep 17 10:54:10.992586 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:10.992578    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-rbac-proxy-cm podName:79ee78ea-aacc-4ecb-a534-f4a8cfd4e349 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:11.992572667 +0000 UTC m=+47.308935799 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nwtnk" (UID: "79ee78ea-aacc-4ecb-a534-f4a8cfd4e349") : configmap "kube-rbac-proxy" not found
Sep 17 10:54:12.000266 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:12.000228    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:12.000650 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:12.000329    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:12.000650 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:12.000393    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 10:54:12.000650 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:12.000412    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 10:54:12.000650 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:12.000446    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-rbac-proxy-cm podName:79ee78ea-aacc-4ecb-a534-f4a8cfd4e349 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:14.000432056 +0000 UTC m=+49.316795207 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nwtnk" (UID: "79ee78ea-aacc-4ecb-a534-f4a8cfd4e349") : configmap "kube-rbac-proxy" not found
Sep 17 10:54:12.000650 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:12.000474    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls podName:79ee78ea-aacc-4ecb-a534-f4a8cfd4e349 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:14.000456693 +0000 UTC m=+49.316819842 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nwtnk" (UID: "79ee78ea-aacc-4ecb-a534-f4a8cfd4e349") : secret "insights-runtime-extractor-tls" not found
Sep 17 10:54:12.067362 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:12.067338    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-phs4b_a02f264e-0ec3-4ea3-b2a8-aca37526f1c3/dns-node-resolver/0.log"
Sep 17 10:54:13.269127 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:13.269095    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rdz5r_f7775776-e097-4507-9a6a-5d21c14ebe07/node-ca/0.log"
Sep 17 10:54:14.014589 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:14.014552    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:14.014772 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:14.014649    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:14.014772 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:14.014726    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 10:54:14.014772 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:14.014730    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 10:54:14.014873 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:14.014776    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-rbac-proxy-cm podName:79ee78ea-aacc-4ecb-a534-f4a8cfd4e349 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:18.014763298 +0000 UTC m=+53.331126444 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nwtnk" (UID: "79ee78ea-aacc-4ecb-a534-f4a8cfd4e349") : configmap "kube-rbac-proxy" not found
Sep 17 10:54:14.014873 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:14.014804    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls podName:79ee78ea-aacc-4ecb-a534-f4a8cfd4e349 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:18.014786849 +0000 UTC m=+53.331149983 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nwtnk" (UID: "79ee78ea-aacc-4ecb-a534-f4a8cfd4e349") : secret "insights-runtime-extractor-tls" not found
Sep 17 10:54:14.073199 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:14.073171    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vrd77_80044c26-9705-435f-a719-4a0e57ac5c75/serve-healthcheck-canary/0.log"
Sep 17 10:54:15.423508 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:15.423449    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:54:15.423508 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:15.423519    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:54:15.424064 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:15.423606    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 10:54:15.424064 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:15.423624    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fb85b79c5-rlhvv: secret "image-registry-tls" not found
Sep 17 10:54:15.424064 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:15.423636    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 10:54:15.424064 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:15.423686    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls podName:3d4e645b-517d-42e1-92d2-107039f0fce9 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:31.423670622 +0000 UTC m=+66.740033775 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls") pod "image-registry-7fb85b79c5-rlhvv" (UID: "3d4e645b-517d-42e1-92d2-107039f0fce9") : secret "image-registry-tls" not found
Sep 17 10:54:15.424064 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:15.423700    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls podName:a3b2ce07-6399-4354-8cdd-be6af04d46a8 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:31.423694501 +0000 UTC m=+66.740057634 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls") pod "dns-default-f466l" (UID: "a3b2ce07-6399-4354-8cdd-be6af04d46a8") : secret "dns-default-metrics-tls" not found
Sep 17 10:54:18.044537 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:18.044501    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:18.045109 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:18.044582    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:18.045109 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:18.044668    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 10:54:18.045109 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:18.044754    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls podName:79ee78ea-aacc-4ecb-a534-f4a8cfd4e349 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:26.04473384 +0000 UTC m=+61.361096980 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nwtnk" (UID: "79ee78ea-aacc-4ecb-a534-f4a8cfd4e349") : secret "insights-runtime-extractor-tls" not found
Sep 17 10:54:18.045109 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:18.045019    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:26.099771 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:26.099710    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:26.102177 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:26.102151    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79ee78ea-aacc-4ecb-a534-f4a8cfd4e349-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nwtnk\" (UID: \"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349\") " pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:26.128219 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:26.128197    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8sqhr\""
Sep 17 10:54:26.137151 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:26.137129    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nwtnk"
Sep 17 10:54:26.320262 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:26.320236    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c559v"
Sep 17 10:54:26.432308 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:26.432233    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nwtnk"]
Sep 17 10:54:26.435289 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:54:26.435246    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ee78ea_aacc_4ecb_a534_f4a8cfd4e349.slice/crio-44d54f50f90ab2cad88631f5a7ac3b205ec045a4d774b180e11b240af32c7ebe WatchSource:0}: Error finding container 44d54f50f90ab2cad88631f5a7ac3b205ec045a4d774b180e11b240af32c7ebe: Status 404 returned error can't find the container with id 44d54f50f90ab2cad88631f5a7ac3b205ec045a4d774b180e11b240af32c7ebe
Sep 17 10:54:27.404974 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:27.404941    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nwtnk" event={"ID":"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349","Type":"ContainerStarted","Data":"b74b8705f3c8ce817de7a60f0a3939a36be115776eab65013c7d4da7757ec829"}
Sep 17 10:54:27.405351 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:27.404995    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nwtnk" event={"ID":"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349","Type":"ContainerStarted","Data":"a89625bf615ad490ae0a4e752ebe9d9722c56aed10efc43f9e988bf98450eedc"}
Sep 17 10:54:27.405351 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:27.405015    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nwtnk" event={"ID":"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349","Type":"ContainerStarted","Data":"44d54f50f90ab2cad88631f5a7ac3b205ec045a4d774b180e11b240af32c7ebe"}
Sep 17 10:54:29.411997 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:29.411958    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nwtnk" event={"ID":"79ee78ea-aacc-4ecb-a534-f4a8cfd4e349","Type":"ContainerStarted","Data":"717c2537266e120d7c4205292f8e8cc32bcb8b8cbd578c5e6099ffcdef91e2b8"}
Sep 17 10:54:29.430233 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:29.430050    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nwtnk" podStartSLOduration=17.341475259 podStartE2EDuration="19.430029518s" totalImagesPullingTime="2.088554259s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 10:54:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:54:29.429859674 +0000 UTC m=+64.746222829" watchObservedRunningTime="2026-09-17 10:54:29.430029518 +0000 UTC m=+64.746392673"
Sep 17 10:54:30.933042 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:30.932998    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs\") pod \"network-metrics-daemon-g7wtx\" (UID: \"c0028c99-f81b-4054-9098-152b859dd374\") " pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:54:30.933042 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:30.933042    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret\") pod \"global-pull-secret-syncer-ddcnf\" (UID: \"909dc7e0-fcc5-4736-8874-8d5206f3f741\") " pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:54:30.935524 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:30.935506    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 10:54:30.935824 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:30.935807    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 10:54:30.945524 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:30.945494    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/909dc7e0-fcc5-4736-8874-8d5206f3f741-original-pull-secret\") pod \"global-pull-secret-syncer-ddcnf\" (UID: \"909dc7e0-fcc5-4736-8874-8d5206f3f741\") " pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:54:30.945615 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:30.945497    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0028c99-f81b-4054-9098-152b859dd374-metrics-certs\") pod \"network-metrics-daemon-g7wtx\" (UID: \"c0028c99-f81b-4054-9098-152b859dd374\") " pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:54:31.033956 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.033933    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wngvt\" (UniqueName: \"kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt\") pod \"network-check-target-fnw2s\" (UID: \"82f7e981-33bb-4353-99bc-1500f39cd125\") " pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:54:31.036474 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.036450    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 10:54:31.046444 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.046424    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 10:54:31.057444 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.057420    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wngvt\" (UniqueName: \"kubernetes.io/projected/82f7e981-33bb-4353-99bc-1500f39cd125-kube-api-access-wngvt\") pod \"network-check-target-fnw2s\" (UID: \"82f7e981-33bb-4353-99bc-1500f39cd125\") " pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:54:31.058571 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.058556    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4chbd\""
Sep 17 10:54:31.062166 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.062149    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ddcnf"
Sep 17 10:54:31.067363 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.067339    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wtx"
Sep 17 10:54:31.341127 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.341101    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ddcnf"]
Sep 17 10:54:31.343758 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.343730    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g7wtx"]
Sep 17 10:54:31.344507 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:54:31.344481    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod909dc7e0_fcc5_4736_8874_8d5206f3f741.slice/crio-ba77e173a0df557f1d0b6ee11bd3fba628d830f52360aad3d52a943b152d7330 WatchSource:0}: Error finding container ba77e173a0df557f1d0b6ee11bd3fba628d830f52360aad3d52a943b152d7330: Status 404 returned error can't find the container with id ba77e173a0df557f1d0b6ee11bd3fba628d830f52360aad3d52a943b152d7330
Sep 17 10:54:31.347012 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:54:31.346988    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0028c99_f81b_4054_9098_152b859dd374.slice/crio-d5bccdf9b24beff27b82d7cbf34fd62a504b9507ef2077749684d5c5c89ae361 WatchSource:0}: Error finding container d5bccdf9b24beff27b82d7cbf34fd62a504b9507ef2077749684d5c5c89ae361: Status 404 returned error can't find the container with id d5bccdf9b24beff27b82d7cbf34fd62a504b9507ef2077749684d5c5c89ae361
Sep 17 10:54:31.350789 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.350771    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gx69l\""
Sep 17 10:54:31.359627 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.359610    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:54:31.417588 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.417560    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g7wtx" event={"ID":"c0028c99-f81b-4054-9098-152b859dd374","Type":"ContainerStarted","Data":"d5bccdf9b24beff27b82d7cbf34fd62a504b9507ef2077749684d5c5c89ae361"}
Sep 17 10:54:31.418734 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.418711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ddcnf" event={"ID":"909dc7e0-fcc5-4736-8874-8d5206f3f741","Type":"ContainerStarted","Data":"ba77e173a0df557f1d0b6ee11bd3fba628d830f52360aad3d52a943b152d7330"}
Sep 17 10:54:31.436119 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.436092    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:54:31.436204 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.436129    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:54:31.438408 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.438379    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3b2ce07-6399-4354-8cdd-be6af04d46a8-metrics-tls\") pod \"dns-default-f466l\" (UID: \"a3b2ce07-6399-4354-8cdd-be6af04d46a8\") " pod="openshift-dns/dns-default-f466l"
Sep 17 10:54:31.438509 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.438496    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls\") pod \"image-registry-7fb85b79c5-rlhvv\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") " pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:54:31.647327 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.647268    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fnw2s"]
Sep 17 10:54:31.650231 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.650211    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n4lnv\""
Sep 17 10:54:31.650328 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:54:31.650215    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82f7e981_33bb_4353_99bc_1500f39cd125.slice/crio-e4d023f1ae19a86f2d5d6552a9ac4ce6a971100d1c06ed6ce8e961c38a6be8f6 WatchSource:0}: Error finding container e4d023f1ae19a86f2d5d6552a9ac4ce6a971100d1c06ed6ce8e961c38a6be8f6: Status 404 returned error can't find the container with id e4d023f1ae19a86f2d5d6552a9ac4ce6a971100d1c06ed6ce8e961c38a6be8f6
Sep 17 10:54:31.658647 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.658628    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:54:31.670370 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.670352    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lfdp8\""
Sep 17 10:54:31.678662 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.678644    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f466l"
Sep 17 10:54:31.935957 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.935862    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fb85b79c5-rlhvv"]
Sep 17 10:54:31.939253 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:54:31.939228    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d4e645b_517d_42e1_92d2_107039f0fce9.slice/crio-2ad8f8f6e0c82d5fd4dafed5a25907576ee05c0a25adc5fb1bb1893ba90f6315 WatchSource:0}: Error finding container 2ad8f8f6e0c82d5fd4dafed5a25907576ee05c0a25adc5fb1bb1893ba90f6315: Status 404 returned error can't find the container with id 2ad8f8f6e0c82d5fd4dafed5a25907576ee05c0a25adc5fb1bb1893ba90f6315
Sep 17 10:54:31.952113 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:31.952092    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f466l"]
Sep 17 10:54:31.955202 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:54:31.955172    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3b2ce07_6399_4354_8cdd_be6af04d46a8.slice/crio-b79303b10a195588979943883258a4e41164a7fbfc5a3e36b01320274fb58cea WatchSource:0}: Error finding container b79303b10a195588979943883258a4e41164a7fbfc5a3e36b01320274fb58cea: Status 404 returned error can't find the container with id b79303b10a195588979943883258a4e41164a7fbfc5a3e36b01320274fb58cea
Sep 17 10:54:32.424739 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:32.424654    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f466l" event={"ID":"a3b2ce07-6399-4354-8cdd-be6af04d46a8","Type":"ContainerStarted","Data":"b79303b10a195588979943883258a4e41164a7fbfc5a3e36b01320274fb58cea"}
Sep 17 10:54:32.427509 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:32.426764    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv" event={"ID":"3d4e645b-517d-42e1-92d2-107039f0fce9","Type":"ContainerStarted","Data":"277e62bc04b348da1effcd59d751cbaed9bb637c3507de40ae75fb24c15e2565"}
Sep 17 10:54:32.427509 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:32.426802    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv" event={"ID":"3d4e645b-517d-42e1-92d2-107039f0fce9","Type":"ContainerStarted","Data":"2ad8f8f6e0c82d5fd4dafed5a25907576ee05c0a25adc5fb1bb1893ba90f6315"}
Sep 17 10:54:32.427509 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:32.427463    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:54:32.429303 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:32.429249    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fnw2s" event={"ID":"82f7e981-33bb-4353-99bc-1500f39cd125","Type":"ContainerStarted","Data":"e4d023f1ae19a86f2d5d6552a9ac4ce6a971100d1c06ed6ce8e961c38a6be8f6"}
Sep 17 10:54:33.437007 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:33.436969    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g7wtx" event={"ID":"c0028c99-f81b-4054-9098-152b859dd374","Type":"ContainerStarted","Data":"0473076af0e07309f956ce6c3b38c5f4c048e96437e9db413aaf210278dda0b9"}
Sep 17 10:54:33.437440 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:33.437032    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g7wtx" event={"ID":"c0028c99-f81b-4054-9098-152b859dd374","Type":"ContainerStarted","Data":"58163d87c15be506d1f0bf9e2293276b93a98a64152ec68d0cf1f350e723f5e2"}
Sep 17 10:54:33.456311 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:33.456258    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv" podStartSLOduration=44.456239333 podStartE2EDuration="44.456239333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:53:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:54:32.455172885 +0000 UTC m=+67.771536042" watchObservedRunningTime="2026-09-17 10:54:33.456239333 +0000 UTC m=+68.772602489"
Sep 17 10:54:33.457465 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:33.457422    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-g7wtx" podStartSLOduration=65.846408943 podStartE2EDuration="1m7.457391626s" totalImagesPullingTime="1.610982683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:53:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:54:33.454662399 +0000 UTC m=+68.771025555" watchObservedRunningTime="2026-09-17 10:54:33.457391626 +0000 UTC m=+68.773754785"
Sep 17 10:54:34.443155 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.443120    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f466l" event={"ID":"a3b2ce07-6399-4354-8cdd-be6af04d46a8","Type":"ContainerStarted","Data":"78d2f55f1d8c1b2381862a064b962004fe140816232f92a333af521881fbabda"}
Sep 17 10:54:34.443654 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.443211    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f466l" event={"ID":"a3b2ce07-6399-4354-8cdd-be6af04d46a8","Type":"ContainerStarted","Data":"05b2cdf8c0c621e5a010b517f2cffb97fc5e6a1db12fd2d7e4f346529872e623"}
Sep 17 10:54:34.443654 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.443601    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-f466l"
Sep 17 10:54:34.471038 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.470993    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-f466l" podStartSLOduration=33.466158023 podStartE2EDuration="35.470978297s" totalImagesPullingTime="2.004820274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:54:34.469329201 +0000 UTC m=+69.785692357" watchObservedRunningTime="2026-09-17 10:54:34.470978297 +0000 UTC m=+69.787341455"
Sep 17 10:54:34.799739 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.799707    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-56ptm"]
Sep 17 10:54:34.802775 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.802745    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-56ptm"
Sep 17 10:54:34.806695 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.806672    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 17 10:54:34.806954 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.806938    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 17 10:54:34.807296 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.807274    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-cp7lj\""
Sep 17 10:54:34.833987 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.833961    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-56ptm"]
Sep 17 10:54:34.834829 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.834807    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg"]
Sep 17 10:54:34.837786 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.837768    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg"
Sep 17 10:54:34.846046 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.846027    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 17 10:54:34.846527 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.846256    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 17 10:54:34.846527 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.846327    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 17 10:54:34.847371 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.847350    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 17 10:54:34.860632 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.860609    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg"]
Sep 17 10:54:34.862439 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.862414    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9c0a4d3d-f882-4df0-abaf-6e0cfe5ec065-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-56ptm\" (UID: \"9c0a4d3d-f882-4df0-abaf-6e0cfe5ec065\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-56ptm"
Sep 17 10:54:34.862548 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.862456    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/938edcc6-5c11-4676-afb7-d54ac437158c-tmp\") pod \"klusterlet-addon-workmgr-86d6fc6f96-57cdg\" (UID: \"938edcc6-5c11-4676-afb7-d54ac437158c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg"
Sep 17 10:54:34.862548 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.862495    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q55s8\" (UniqueName: \"kubernetes.io/projected/938edcc6-5c11-4676-afb7-d54ac437158c-kube-api-access-q55s8\") pod \"klusterlet-addon-workmgr-86d6fc6f96-57cdg\" (UID: \"938edcc6-5c11-4676-afb7-d54ac437158c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg"
Sep 17 10:54:34.862548 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.862529    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9c0a4d3d-f882-4df0-abaf-6e0cfe5ec065-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-56ptm\" (UID: \"9c0a4d3d-f882-4df0-abaf-6e0cfe5ec065\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-56ptm"
Sep 17 10:54:34.862696 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.862553    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/938edcc6-5c11-4676-afb7-d54ac437158c-klusterlet-config\") pod \"klusterlet-addon-workmgr-86d6fc6f96-57cdg\" (UID: \"938edcc6-5c11-4676-afb7-d54ac437158c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg"
Sep 17 10:54:34.913373 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.913347    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7fb85b79c5-rlhvv"]
Sep 17 10:54:34.963870 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.963836    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9c0a4d3d-f882-4df0-abaf-6e0cfe5ec065-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-56ptm\" (UID: \"9c0a4d3d-f882-4df0-abaf-6e0cfe5ec065\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-56ptm"
Sep 17 10:54:34.964038 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.963881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/938edcc6-5c11-4676-afb7-d54ac437158c-klusterlet-config\") pod \"klusterlet-addon-workmgr-86d6fc6f96-57cdg\" (UID: \"938edcc6-5c11-4676-afb7-d54ac437158c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg"
Sep 17 10:54:34.964038 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.963967    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9c0a4d3d-f882-4df0-abaf-6e0cfe5ec065-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-56ptm\" (UID: \"9c0a4d3d-f882-4df0-abaf-6e0cfe5ec065\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-56ptm"
Sep 17 10:54:34.964038 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.963996    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/938edcc6-5c11-4676-afb7-d54ac437158c-tmp\") pod \"klusterlet-addon-workmgr-86d6fc6f96-57cdg\" (UID: \"938edcc6-5c11-4676-afb7-d54ac437158c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg"
Sep 17 10:54:34.964198 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.964035    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q55s8\" (UniqueName: \"kubernetes.io/projected/938edcc6-5c11-4676-afb7-d54ac437158c-kube-api-access-q55s8\") pod \"klusterlet-addon-workmgr-86d6fc6f96-57cdg\" (UID: \"938edcc6-5c11-4676-afb7-d54ac437158c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg"
Sep 17 10:54:34.964631 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.964581    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/938edcc6-5c11-4676-afb7-d54ac437158c-tmp\") pod \"klusterlet-addon-workmgr-86d6fc6f96-57cdg\" (UID: \"938edcc6-5c11-4676-afb7-d54ac437158c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg"
Sep 17 10:54:34.964631 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.964599    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9c0a4d3d-f882-4df0-abaf-6e0cfe5ec065-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-56ptm\" (UID: \"9c0a4d3d-f882-4df0-abaf-6e0cfe5ec065\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-56ptm"
Sep 17 10:54:34.966912 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.966874    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9c0a4d3d-f882-4df0-abaf-6e0cfe5ec065-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-56ptm\" (UID: \"9c0a4d3d-f882-4df0-abaf-6e0cfe5ec065\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-56ptm"
Sep 17 10:54:34.967031 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.966916    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/938edcc6-5c11-4676-afb7-d54ac437158c-klusterlet-config\") pod \"klusterlet-addon-workmgr-86d6fc6f96-57cdg\" (UID: \"938edcc6-5c11-4676-afb7-d54ac437158c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg"
Sep 17 10:54:34.969781 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.969758    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86d9d67866-242j2"]
Sep 17 10:54:34.973147 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.973129    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:34.988116 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.988093    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q55s8\" (UniqueName: \"kubernetes.io/projected/938edcc6-5c11-4676-afb7-d54ac437158c-kube-api-access-q55s8\") pod \"klusterlet-addon-workmgr-86d6fc6f96-57cdg\" (UID: \"938edcc6-5c11-4676-afb7-d54ac437158c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg"
Sep 17 10:54:34.996517 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:34.996497    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86d9d67866-242j2"]
Sep 17 10:54:35.064673 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.064596    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/57c6e2bf-09ef-4558-99e6-a4befe6d8505-image-registry-private-configuration\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.064673 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.064649    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/57c6e2bf-09ef-4558-99e6-a4befe6d8505-trusted-ca\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.064839 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.064683    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/57c6e2bf-09ef-4558-99e6-a4befe6d8505-ca-trust-extracted\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.064839 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.064724    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/57c6e2bf-09ef-4558-99e6-a4befe6d8505-installation-pull-secrets\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.064839 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.064759    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/57c6e2bf-09ef-4558-99e6-a4befe6d8505-registry-tls\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.064839 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.064786    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4w4t\" (UniqueName: \"kubernetes.io/projected/57c6e2bf-09ef-4558-99e6-a4befe6d8505-kube-api-access-w4w4t\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.064839 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.064822    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/57c6e2bf-09ef-4558-99e6-a4befe6d8505-bound-sa-token\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.065096 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.064862    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/57c6e2bf-09ef-4558-99e6-a4befe6d8505-registry-certificates\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.113358 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.113331    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-56ptm"
Sep 17 10:54:35.149284 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.149255    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg"
Sep 17 10:54:35.166210 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.166186    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/57c6e2bf-09ef-4558-99e6-a4befe6d8505-installation-pull-secrets\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.166317 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.166224    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/57c6e2bf-09ef-4558-99e6-a4befe6d8505-registry-tls\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.166317 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.166244    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4w4t\" (UniqueName: \"kubernetes.io/projected/57c6e2bf-09ef-4558-99e6-a4befe6d8505-kube-api-access-w4w4t\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.166317 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.166265    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/57c6e2bf-09ef-4558-99e6-a4befe6d8505-bound-sa-token\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.166317 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.166297    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/57c6e2bf-09ef-4558-99e6-a4befe6d8505-registry-certificates\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.166507 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.166473    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/57c6e2bf-09ef-4558-99e6-a4befe6d8505-image-registry-private-configuration\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.166560 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.166524    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/57c6e2bf-09ef-4558-99e6-a4befe6d8505-trusted-ca\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.166612 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.166560    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/57c6e2bf-09ef-4558-99e6-a4befe6d8505-ca-trust-extracted\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.167221 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.167016    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/57c6e2bf-09ef-4558-99e6-a4befe6d8505-ca-trust-extracted\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.167317 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.167288    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/57c6e2bf-09ef-4558-99e6-a4befe6d8505-registry-certificates\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.167688 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.167660    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/57c6e2bf-09ef-4558-99e6-a4befe6d8505-trusted-ca\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.169121 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.169098    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/57c6e2bf-09ef-4558-99e6-a4befe6d8505-installation-pull-secrets\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.169205 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.169106    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/57c6e2bf-09ef-4558-99e6-a4befe6d8505-registry-tls\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.169321 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.169303    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/57c6e2bf-09ef-4558-99e6-a4befe6d8505-image-registry-private-configuration\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.178729 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.178672    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/57c6e2bf-09ef-4558-99e6-a4befe6d8505-bound-sa-token\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.180140 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.180122    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4w4t\" (UniqueName: \"kubernetes.io/projected/57c6e2bf-09ef-4558-99e6-a4befe6d8505-kube-api-access-w4w4t\") pod \"image-registry-86d9d67866-242j2\" (UID: \"57c6e2bf-09ef-4558-99e6-a4befe6d8505\") " pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:35.286593 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:35.286560    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:37.344479 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:37.344308    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg"]
Sep 17 10:54:37.348742 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:54:37.348714    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod938edcc6_5c11_4676_afb7_d54ac437158c.slice/crio-e08f4cba191118337baf6c49faccd2d1ce2e067ad8a7872d5f29aecc3a238c9c WatchSource:0}: Error finding container e08f4cba191118337baf6c49faccd2d1ce2e067ad8a7872d5f29aecc3a238c9c: Status 404 returned error can't find the container with id e08f4cba191118337baf6c49faccd2d1ce2e067ad8a7872d5f29aecc3a238c9c
Sep 17 10:54:37.384597 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:37.384569    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86d9d67866-242j2"]
Sep 17 10:54:37.386846 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:54:37.386813    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57c6e2bf_09ef_4558_99e6_a4befe6d8505.slice/crio-500daf62bb1ea7f40f8d525df09b269521dbf39f7cf96029e7de14683a2288a3 WatchSource:0}: Error finding container 500daf62bb1ea7f40f8d525df09b269521dbf39f7cf96029e7de14683a2288a3: Status 404 returned error can't find the container with id 500daf62bb1ea7f40f8d525df09b269521dbf39f7cf96029e7de14683a2288a3
Sep 17 10:54:37.399485 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:37.399460    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-56ptm"]
Sep 17 10:54:37.402410 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:54:37.402385    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c0a4d3d_f882_4df0_abaf_6e0cfe5ec065.slice/crio-2ec91eb71283f32d77a01e0d8dace443067d355f33e1f5f4f62feea88e4a9917 WatchSource:0}: Error finding container 2ec91eb71283f32d77a01e0d8dace443067d355f33e1f5f4f62feea88e4a9917: Status 404 returned error can't find the container with id 2ec91eb71283f32d77a01e0d8dace443067d355f33e1f5f4f62feea88e4a9917
Sep 17 10:54:37.455649 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:37.455620    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86d9d67866-242j2" event={"ID":"57c6e2bf-09ef-4558-99e6-a4befe6d8505","Type":"ContainerStarted","Data":"500daf62bb1ea7f40f8d525df09b269521dbf39f7cf96029e7de14683a2288a3"}
Sep 17 10:54:37.457499 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:37.457470    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ddcnf" event={"ID":"909dc7e0-fcc5-4736-8874-8d5206f3f741","Type":"ContainerStarted","Data":"ab84e141b16746fa356bf0faa4af398e5702e27afccba370fb5b0863760ab0f2"}
Sep 17 10:54:37.459180 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:37.459158    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-56ptm" event={"ID":"9c0a4d3d-f882-4df0-abaf-6e0cfe5ec065","Type":"ContainerStarted","Data":"2ec91eb71283f32d77a01e0d8dace443067d355f33e1f5f4f62feea88e4a9917"}
Sep 17 10:54:37.460790 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:37.460761    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fnw2s" event={"ID":"82f7e981-33bb-4353-99bc-1500f39cd125","Type":"ContainerStarted","Data":"9fb2676b77cfda6ffb2d5cb105c05b21211f05a7e716d2468d293e818a4bb219"}
Sep 17 10:54:37.460891 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:37.460824    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:54:37.462023 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:37.462000    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg" event={"ID":"938edcc6-5c11-4676-afb7-d54ac437158c","Type":"ContainerStarted","Data":"e08f4cba191118337baf6c49faccd2d1ce2e067ad8a7872d5f29aecc3a238c9c"}
Sep 17 10:54:37.483249 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:37.483082    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ddcnf" podStartSLOduration=65.736692673 podStartE2EDuration="1m11.483067171s" totalImagesPullingTime="5.746374498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 10:53:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:54:37.481854636 +0000 UTC m=+72.798217812" watchObservedRunningTime="2026-09-17 10:54:37.483067171 +0000 UTC m=+72.799430326"
Sep 17 10:54:37.504257 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:37.504208    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fnw2s" podStartSLOduration=66.07646992 podStartE2EDuration="1m11.50418886s" totalImagesPullingTime="5.42771894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:53:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:54:37.5036617 +0000 UTC m=+72.820024854" watchObservedRunningTime="2026-09-17 10:54:37.50418886 +0000 UTC m=+72.820552016"
Sep 17 10:54:38.468853 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:38.468807    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86d9d67866-242j2" event={"ID":"57c6e2bf-09ef-4558-99e6-a4befe6d8505","Type":"ContainerStarted","Data":"bafc2859dcc050ee40f9e2c86b5c8423f7588a414e5f4caa7f42e666c00364ad"}
Sep 17 10:54:39.473137 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:39.473104    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:54:40.479050 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:40.479011    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-56ptm" event={"ID":"9c0a4d3d-f882-4df0-abaf-6e0cfe5ec065","Type":"ContainerStarted","Data":"6a8ac9e8f3aa6426c9e65f317c75a82f14bba834cb3a7d64c3bfef53790d23fd"}
Sep 17 10:54:40.498233 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:40.498186    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-56ptm" podStartSLOduration=4.042187753 podStartE2EDuration="6.498166655s" totalImagesPullingTime="2.455978902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:54:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:54:40.497274227 +0000 UTC m=+75.813637383" watchObservedRunningTime="2026-09-17 10:54:40.498166655 +0000 UTC m=+75.814529811"
Sep 17 10:54:40.498429 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:40.498318    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-86d9d67866-242j2" podStartSLOduration=6.498309429 podStartE2EDuration="6.498309429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:54:38.508424477 +0000 UTC m=+73.824787634" watchObservedRunningTime="2026-09-17 10:54:40.498309429 +0000 UTC m=+75.814672582"
Sep 17 10:54:41.483634 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.483531    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg" event={"ID":"938edcc6-5c11-4676-afb7-d54ac437158c","Type":"ContainerStarted","Data":"ed567614f32f5d26bdcd4df8e0c644a2f2259b584dcdeea3419afc0924f15b21"}
Sep 17 10:54:41.484056 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.483750    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg"
Sep 17 10:54:41.485395 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.485372    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg"
Sep 17 10:54:41.506356 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.506306    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86d6fc6f96-57cdg" podStartSLOduration=3.73736456 podStartE2EDuration="7.506291645s" totalImagesPullingTime="3.768927085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:54:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:54:41.50515897 +0000 UTC m=+76.821522126" watchObservedRunningTime="2026-09-17 10:54:41.506291645 +0000 UTC m=+76.822654799"
Sep 17 10:54:41.589998 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.589971    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"]
Sep 17 10:54:41.591970 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.591954    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"
Sep 17 10:54:41.595442 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.595422    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 17 10:54:41.595562 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.595491    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 17 10:54:41.595635 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.595579    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 10:54:41.595857 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.595841    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jb7n6\""
Sep 17 10:54:41.595939 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.595892    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 10:54:41.595939 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.595928    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 10:54:41.606650 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.606631    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"]
Sep 17 10:54:41.721571 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.721539    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ca4015e-76d6-45ce-89ba-5dacd3f6aac4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-vq86r\" (UID: \"8ca4015e-76d6-45ce-89ba-5dacd3f6aac4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"
Sep 17 10:54:41.721713 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.721579    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8ca4015e-76d6-45ce-89ba-5dacd3f6aac4-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-vq86r\" (UID: \"8ca4015e-76d6-45ce-89ba-5dacd3f6aac4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"
Sep 17 10:54:41.721713 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.721607    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvgmm\" (UniqueName: \"kubernetes.io/projected/8ca4015e-76d6-45ce-89ba-5dacd3f6aac4-kube-api-access-fvgmm\") pod \"prometheus-operator-68bcd7f987-vq86r\" (UID: \"8ca4015e-76d6-45ce-89ba-5dacd3f6aac4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"
Sep 17 10:54:41.721713 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.721637    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ca4015e-76d6-45ce-89ba-5dacd3f6aac4-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-vq86r\" (UID: \"8ca4015e-76d6-45ce-89ba-5dacd3f6aac4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"
Sep 17 10:54:41.822611 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.822546    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ca4015e-76d6-45ce-89ba-5dacd3f6aac4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-vq86r\" (UID: \"8ca4015e-76d6-45ce-89ba-5dacd3f6aac4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"
Sep 17 10:54:41.822611 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.822582    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8ca4015e-76d6-45ce-89ba-5dacd3f6aac4-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-vq86r\" (UID: \"8ca4015e-76d6-45ce-89ba-5dacd3f6aac4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"
Sep 17 10:54:41.822750 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:41.822685    2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 17 10:54:41.822750 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.822710    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvgmm\" (UniqueName: \"kubernetes.io/projected/8ca4015e-76d6-45ce-89ba-5dacd3f6aac4-kube-api-access-fvgmm\") pod \"prometheus-operator-68bcd7f987-vq86r\" (UID: \"8ca4015e-76d6-45ce-89ba-5dacd3f6aac4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"
Sep 17 10:54:41.822750 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:41.822742    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ca4015e-76d6-45ce-89ba-5dacd3f6aac4-prometheus-operator-tls podName:8ca4015e-76d6-45ce-89ba-5dacd3f6aac4 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:42.322723339 +0000 UTC m=+77.639086477 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/8ca4015e-76d6-45ce-89ba-5dacd3f6aac4-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-vq86r" (UID: "8ca4015e-76d6-45ce-89ba-5dacd3f6aac4") : secret "prometheus-operator-tls" not found
Sep 17 10:54:41.822880 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.822776    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ca4015e-76d6-45ce-89ba-5dacd3f6aac4-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-vq86r\" (UID: \"8ca4015e-76d6-45ce-89ba-5dacd3f6aac4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"
Sep 17 10:54:41.823408 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.823345    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ca4015e-76d6-45ce-89ba-5dacd3f6aac4-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-vq86r\" (UID: \"8ca4015e-76d6-45ce-89ba-5dacd3f6aac4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"
Sep 17 10:54:41.825137 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.825114    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ca4015e-76d6-45ce-89ba-5dacd3f6aac4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-vq86r\" (UID: \"8ca4015e-76d6-45ce-89ba-5dacd3f6aac4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"
Sep 17 10:54:41.836636 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:41.836607    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvgmm\" (UniqueName: \"kubernetes.io/projected/8ca4015e-76d6-45ce-89ba-5dacd3f6aac4-kube-api-access-fvgmm\") pod \"prometheus-operator-68bcd7f987-vq86r\" (UID: \"8ca4015e-76d6-45ce-89ba-5dacd3f6aac4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"
Sep 17 10:54:42.327045 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:42.327003    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8ca4015e-76d6-45ce-89ba-5dacd3f6aac4-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-vq86r\" (UID: \"8ca4015e-76d6-45ce-89ba-5dacd3f6aac4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"
Sep 17 10:54:42.329421 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:42.329400    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8ca4015e-76d6-45ce-89ba-5dacd3f6aac4-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-vq86r\" (UID: \"8ca4015e-76d6-45ce-89ba-5dacd3f6aac4\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"
Sep 17 10:54:42.500915 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:42.500871    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"
Sep 17 10:54:42.773470 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:42.773441    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-vq86r"]
Sep 17 10:54:42.776951 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:54:42.776926    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ca4015e_76d6_45ce_89ba_5dacd3f6aac4.slice/crio-06895d38644b8e430a9c8f2bdc002fd5ad3d672346b436f463273097e1fd67da WatchSource:0}: Error finding container 06895d38644b8e430a9c8f2bdc002fd5ad3d672346b436f463273097e1fd67da: Status 404 returned error can't find the container with id 06895d38644b8e430a9c8f2bdc002fd5ad3d672346b436f463273097e1fd67da
Sep 17 10:54:43.490960 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:43.490918    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r" event={"ID":"8ca4015e-76d6-45ce-89ba-5dacd3f6aac4","Type":"ContainerStarted","Data":"06895d38644b8e430a9c8f2bdc002fd5ad3d672346b436f463273097e1fd67da"}
Sep 17 10:54:44.496719 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:44.496685    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r" event={"ID":"8ca4015e-76d6-45ce-89ba-5dacd3f6aac4","Type":"ContainerStarted","Data":"bc8db2ad79a25a1cb61666061d4f99ac56563514e2b6293a080141a724eee1b6"}
Sep 17 10:54:44.497098 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:44.496743    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r" event={"ID":"8ca4015e-76d6-45ce-89ba-5dacd3f6aac4","Type":"ContainerStarted","Data":"df7adf29df125a6e61177f48af61b123ee284cfb50a8ae2b38896e3bef662c02"}
Sep 17 10:54:44.524299 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:44.524249    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vq86r" podStartSLOduration=1.937138992 podStartE2EDuration="3.524233325s" totalImagesPullingTime="1.587094333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:54:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:54:44.523500577 +0000 UTC m=+79.839863731" watchObservedRunningTime="2026-09-17 10:54:44.524233325 +0000 UTC m=+79.840596479"
Sep 17 10:54:45.452543 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:45.452516    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-f466l"
Sep 17 10:54:46.056154 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.056118    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"]
Sep 17 10:54:46.059329 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.059313    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"]
Sep 17 10:54:46.059447 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.059429    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"
Sep 17 10:54:46.062150 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.062128    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 17 10:54:46.062278 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.062179    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 17 10:54:46.062278 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.062209    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-6chjd\""
Sep 17 10:54:46.062389 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.062329    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.065388 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.065369    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 17 10:54:46.065388 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.065384    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 17 10:54:46.065592 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.065580    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fgfc5\""
Sep 17 10:54:46.068301 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.068286    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 17 10:54:46.089585 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.089563    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"]
Sep 17 10:54:46.090527 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.090506    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"]
Sep 17 10:54:46.091350 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.091333    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r9tvg"]
Sep 17 10:54:46.100116 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.100099    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.103159 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.103142    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 10:54:46.103254 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.103173    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 10:54:46.103310 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.103258    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kt2bx\""
Sep 17 10:54:46.103981 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.103960    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 10:54:46.152222 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152201    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxsgw\" (UniqueName: \"kubernetes.io/projected/51364bda-4f67-4195-b557-43e2fb6726ca-kube-api-access-cxsgw\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.152323 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152228    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51364bda-4f67-4195-b557-43e2fb6726ca-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.152323 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152247    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.152323 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152266    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-24xnl\" (UID: \"b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"
Sep 17 10:54:46.152440 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152319    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/51364bda-4f67-4195-b557-43e2fb6726ca-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.152440 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-metrics-client-ca\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.152440 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-tls\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.152440 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152382    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51364bda-4f67-4195-b557-43e2fb6726ca-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.152440 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152403    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-24xnl\" (UID: \"b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"
Sep 17 10:54:46.152440 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152426    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-24xnl\" (UID: \"b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"
Sep 17 10:54:46.152614 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152493    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-accelerators-collector-config\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.152614 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152526    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-wtmp\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.152614 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152546    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf57f\" (UniqueName: \"kubernetes.io/projected/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-kube-api-access-sf57f\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.152614 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152566    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/51364bda-4f67-4195-b557-43e2fb6726ca-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.152614 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152601    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-textfile\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.152761 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152631    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-root\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.152761 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152648    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgc7t\" (UniqueName: \"kubernetes.io/projected/b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c-kube-api-access-zgc7t\") pod \"openshift-state-metrics-67dcf8c979-24xnl\" (UID: \"b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"
Sep 17 10:54:46.152761 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152664    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-sys\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.152761 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.152691    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/51364bda-4f67-4195-b557-43e2fb6726ca-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.253837 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.253805    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-sys\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.253986 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.253841    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/51364bda-4f67-4195-b557-43e2fb6726ca-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.253986 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.253865    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxsgw\" (UniqueName: \"kubernetes.io/projected/51364bda-4f67-4195-b557-43e2fb6726ca-kube-api-access-cxsgw\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.253986 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.253946    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-sys\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.253986 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.253971    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51364bda-4f67-4195-b557-43e2fb6726ca-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.254208 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.254000    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.254208 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.254021    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-24xnl\" (UID: \"b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"
Sep 17 10:54:46.254208 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.254038    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/51364bda-4f67-4195-b557-43e2fb6726ca-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.254208 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.254055    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-metrics-client-ca\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.254208 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.254069    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-tls\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.254208 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.254086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51364bda-4f67-4195-b557-43e2fb6726ca-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.254208 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.254120    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-24xnl\" (UID: \"b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"
Sep 17 10:54:46.254634 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:46.254209    2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 17 10:54:46.254634 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:54:46.254301    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-tls podName:c6048ca1-2e34-4de8-b8bd-4661504dc4e4 nodeName:}" failed. No retries permitted until 2026-09-17 10:54:46.75428185 +0000 UTC m=+82.070644987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-tls") pod "node-exporter-r9tvg" (UID: "c6048ca1-2e34-4de8-b8bd-4661504dc4e4") : secret "node-exporter-tls" not found
Sep 17 10:54:46.254634 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.254594    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-24xnl\" (UID: \"b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"
Sep 17 10:54:46.254790 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.254666    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-accelerators-collector-config\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.255025 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.254830    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51364bda-4f67-4195-b557-43e2fb6726ca-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.255025 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.254810    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-wtmp\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.255025 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.254892    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf57f\" (UniqueName: \"kubernetes.io/projected/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-kube-api-access-sf57f\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.255025 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.254929    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/51364bda-4f67-4195-b557-43e2fb6726ca-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.255025 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.254941    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/51364bda-4f67-4195-b557-43e2fb6726ca-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.255025 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.254972    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-metrics-client-ca\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.255025 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.254992    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-textfile\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.255385 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.255033    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-root\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.255385 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.255065    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgc7t\" (UniqueName: \"kubernetes.io/projected/b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c-kube-api-access-zgc7t\") pod \"openshift-state-metrics-67dcf8c979-24xnl\" (UID: \"b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"
Sep 17 10:54:46.255601 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.255573    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-textfile\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.255684 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.255660    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/51364bda-4f67-4195-b557-43e2fb6726ca-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.255835 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.255814    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-wtmp\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.255949 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.255832    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-24xnl\" (UID: \"b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"
Sep 17 10:54:46.255949 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.255924    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-root\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.256115 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.256089    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-accelerators-collector-config\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.256969 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.256949    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/51364bda-4f67-4195-b557-43e2fb6726ca-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.257074 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.257009    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51364bda-4f67-4195-b557-43e2fb6726ca-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.257440 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.257423    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-24xnl\" (UID: \"b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"
Sep 17 10:54:46.257686 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.257670    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-24xnl\" (UID: \"b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"
Sep 17 10:54:46.258021 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.258002    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.263492 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.263469    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxsgw\" (UniqueName: \"kubernetes.io/projected/51364bda-4f67-4195-b557-43e2fb6726ca-kube-api-access-cxsgw\") pod \"kube-state-metrics-59fbdd77d-9s4c2\" (UID: \"51364bda-4f67-4195-b557-43e2fb6726ca\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.265092 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.265072    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgc7t\" (UniqueName: \"kubernetes.io/projected/b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c-kube-api-access-zgc7t\") pod \"openshift-state-metrics-67dcf8c979-24xnl\" (UID: \"b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"
Sep 17 10:54:46.265424 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.265405    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf57f\" (UniqueName: \"kubernetes.io/projected/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-kube-api-access-sf57f\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.369594 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.369545    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"
Sep 17 10:54:46.374542 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.374530    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"
Sep 17 10:54:46.644380 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.644295    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl"]
Sep 17 10:54:46.647214 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:54:46.647187    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9afe1f0_74e7_4ea4_b8dc_e872d34dcc0c.slice/crio-c2ec0966dd60c1fa47aa003708771624dc3f30b3676eb9ee3393d9b059e07e83 WatchSource:0}: Error finding container c2ec0966dd60c1fa47aa003708771624dc3f30b3676eb9ee3393d9b059e07e83: Status 404 returned error can't find the container with id c2ec0966dd60c1fa47aa003708771624dc3f30b3676eb9ee3393d9b059e07e83
Sep 17 10:54:46.648298 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.648276    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2"]
Sep 17 10:54:46.651122 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:54:46.651102    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51364bda_4f67_4195_b557_43e2fb6726ca.slice/crio-378a60884bc13cbcc8dc31c40c7285bc2ab679a09429237623d88a7892bc27d9 WatchSource:0}: Error finding container 378a60884bc13cbcc8dc31c40c7285bc2ab679a09429237623d88a7892bc27d9: Status 404 returned error can't find the container with id 378a60884bc13cbcc8dc31c40c7285bc2ab679a09429237623d88a7892bc27d9
Sep 17 10:54:46.758783 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.758764    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-tls\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:46.761005 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:46.760983    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c6048ca1-2e34-4de8-b8bd-4661504dc4e4-node-exporter-tls\") pod \"node-exporter-r9tvg\" (UID: \"c6048ca1-2e34-4de8-b8bd-4661504dc4e4\") " pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:47.007813 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:47.007780    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r9tvg"
Sep 17 10:54:47.018422 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:54:47.018394    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6048ca1_2e34_4de8_b8bd_4661504dc4e4.slice/crio-cd3cdbde7830952112db73015d069c5fae6608c5ac78f069c6ea0bfe1fee52f9 WatchSource:0}: Error finding container cd3cdbde7830952112db73015d069c5fae6608c5ac78f069c6ea0bfe1fee52f9: Status 404 returned error can't find the container with id cd3cdbde7830952112db73015d069c5fae6608c5ac78f069c6ea0bfe1fee52f9
Sep 17 10:54:47.510613 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:47.510559    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl" event={"ID":"b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c","Type":"ContainerStarted","Data":"73fb13399fbb159a36874f1ec5fa5d2eafafdf2bd60e57452142f30334313f11"}
Sep 17 10:54:47.511123 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:47.510640    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl" event={"ID":"b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c","Type":"ContainerStarted","Data":"de4be23f84810fac3919676c4946500f19de99874b906883912bef8f00d4687b"}
Sep 17 10:54:47.511123 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:47.510669    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl" event={"ID":"b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c","Type":"ContainerStarted","Data":"c2ec0966dd60c1fa47aa003708771624dc3f30b3676eb9ee3393d9b059e07e83"}
Sep 17 10:54:47.512129 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:47.512100    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9tvg" event={"ID":"c6048ca1-2e34-4de8-b8bd-4661504dc4e4","Type":"ContainerStarted","Data":"cd3cdbde7830952112db73015d069c5fae6608c5ac78f069c6ea0bfe1fee52f9"}
Sep 17 10:54:47.513690 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:47.513617    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2" event={"ID":"51364bda-4f67-4195-b557-43e2fb6726ca","Type":"ContainerStarted","Data":"378a60884bc13cbcc8dc31c40c7285bc2ab679a09429237623d88a7892bc27d9"}
Sep 17 10:54:48.521247 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:48.521121    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2" event={"ID":"51364bda-4f67-4195-b557-43e2fb6726ca","Type":"ContainerStarted","Data":"34ae0171cbabf8e2427d19058ba102685c4b42dc0b8373a033748bc951f7835e"}
Sep 17 10:54:48.524213 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:48.524179    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl" event={"ID":"b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c","Type":"ContainerStarted","Data":"0b54dc774f5bf21554fd78430dce1e10b9c4a19e4c56de751dfb988b9b0ac94b"}
Sep 17 10:54:48.528406 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:48.527422    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9tvg" event={"ID":"c6048ca1-2e34-4de8-b8bd-4661504dc4e4","Type":"ContainerStarted","Data":"625c4553886734ec7dd862c44ef7a58dabf95a6b8edf30922d2fa5b9a53a453d"}
Sep 17 10:54:48.562195 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:48.562137    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-24xnl" podStartSLOduration=0.920369115 podStartE2EDuration="2.562113991s" totalImagesPullingTime="1.641744876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:54:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:54:48.553592971 +0000 UTC m=+83.869956128" watchObservedRunningTime="2026-09-17 10:54:48.562113991 +0000 UTC m=+83.878477149"
Sep 17 10:54:49.532610 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:49.532578    2564 generic.go:356] "Generic (PLEG): container finished" podID="c6048ca1-2e34-4de8-b8bd-4661504dc4e4" containerID="625c4553886734ec7dd862c44ef7a58dabf95a6b8edf30922d2fa5b9a53a453d" exitCode=0
Sep 17 10:54:49.533036 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:49.532654    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9tvg" event={"ID":"c6048ca1-2e34-4de8-b8bd-4661504dc4e4","Type":"ContainerDied","Data":"625c4553886734ec7dd862c44ef7a58dabf95a6b8edf30922d2fa5b9a53a453d"}
Sep 17 10:54:49.535188 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:49.535163    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2" event={"ID":"51364bda-4f67-4195-b557-43e2fb6726ca","Type":"ContainerStarted","Data":"3ac3e731c0d89b011ae3a1f56462019c104f9ab975492d9d32027239ef0571b8"}
Sep 17 10:54:49.535284 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:49.535219    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2" event={"ID":"51364bda-4f67-4195-b557-43e2fb6726ca","Type":"ContainerStarted","Data":"999add0cf79d036dad096447659c5d0d0f23e0f3d8062022e34eed87f4885154"}
Sep 17 10:54:49.575032 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:49.574990    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-9s4c2" podStartSLOduration=1.826572895 podStartE2EDuration="3.574972997s" totalImagesPullingTime="1.748400102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:54:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:54:49.574259311 +0000 UTC m=+84.890622469" watchObservedRunningTime="2026-09-17 10:54:49.574972997 +0000 UTC m=+84.891336153"
Sep 17 10:54:50.545656 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.545617    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9tvg" event={"ID":"c6048ca1-2e34-4de8-b8bd-4661504dc4e4","Type":"ContainerStarted","Data":"0e5f74799b61a4449f3f66e2d3fe325a5cae6a2a8991cca97603179acce9f3e9"}
Sep 17 10:54:50.546052 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.545690    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9tvg" event={"ID":"c6048ca1-2e34-4de8-b8bd-4661504dc4e4","Type":"ContainerStarted","Data":"372123cca53c20e472ee34dc8f8274243874b0e923ff07ca544b7227889d01c0"}
Sep 17 10:54:50.573417 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.573369    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r9tvg" podStartSLOduration=3.098582527 podStartE2EDuration="4.573350916s" totalImagesPullingTime="1.37866682s" totalInitContainerRuntime="96.101569ms" isStatefulPod=true podCreationTimestamp="2026-09-17 10:54:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:54:50.57300203 +0000 UTC m=+85.889365189" watchObservedRunningTime="2026-09-17 10:54:50.573350916 +0000 UTC m=+85.889714070"
Sep 17 10:54:50.585025 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.585000    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"]
Sep 17 10:54:50.588181 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.588162    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.592193 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.592172    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-5gp0l5jr1duqc\""
Sep 17 10:54:50.592531 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.592510    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 17 10:54:50.592641 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.592598    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 17 10:54:50.592641 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.592616    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 17 10:54:50.592977 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.592961    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5w7gg\""
Sep 17 10:54:50.593034 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.593020    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 17 10:54:50.602816 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.602793    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"]
Sep 17 10:54:50.695772 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.695744    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/deaaeeb5-cb5e-45fa-a284-07052b47268e-metrics-server-audit-profiles\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.695772 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.695774    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/deaaeeb5-cb5e-45fa-a284-07052b47268e-client-ca-bundle\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.695974 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.695841    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/deaaeeb5-cb5e-45fa-a284-07052b47268e-secret-metrics-server-client-certs\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.695974 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.695938    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcdgm\" (UniqueName: \"kubernetes.io/projected/deaaeeb5-cb5e-45fa-a284-07052b47268e-kube-api-access-vcdgm\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.696066 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.695974    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/deaaeeb5-cb5e-45fa-a284-07052b47268e-secret-metrics-server-tls\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.696129 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.696114    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deaaeeb5-cb5e-45fa-a284-07052b47268e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.696221 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.696206    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/deaaeeb5-cb5e-45fa-a284-07052b47268e-audit-log\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.797104 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.797031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/deaaeeb5-cb5e-45fa-a284-07052b47268e-metrics-server-audit-profiles\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.797104 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.797071    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/deaaeeb5-cb5e-45fa-a284-07052b47268e-client-ca-bundle\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.797314 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.797105    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/deaaeeb5-cb5e-45fa-a284-07052b47268e-secret-metrics-server-client-certs\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.797314 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.797131    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcdgm\" (UniqueName: \"kubernetes.io/projected/deaaeeb5-cb5e-45fa-a284-07052b47268e-kube-api-access-vcdgm\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.797314 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.797156    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/deaaeeb5-cb5e-45fa-a284-07052b47268e-secret-metrics-server-tls\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.797314 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.797198    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deaaeeb5-cb5e-45fa-a284-07052b47268e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.797314 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.797252    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/deaaeeb5-cb5e-45fa-a284-07052b47268e-audit-log\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.797670 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.797635    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/deaaeeb5-cb5e-45fa-a284-07052b47268e-audit-log\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.798013 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.797988    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/deaaeeb5-cb5e-45fa-a284-07052b47268e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.798140 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.798125    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/deaaeeb5-cb5e-45fa-a284-07052b47268e-metrics-server-audit-profiles\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.799696 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.799674    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/deaaeeb5-cb5e-45fa-a284-07052b47268e-client-ca-bundle\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.799796 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.799690    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/deaaeeb5-cb5e-45fa-a284-07052b47268e-secret-metrics-server-tls\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.799796 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.799787    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/deaaeeb5-cb5e-45fa-a284-07052b47268e-secret-metrics-server-client-certs\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.832169 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.832150    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcdgm\" (UniqueName: \"kubernetes.io/projected/deaaeeb5-cb5e-45fa-a284-07052b47268e-kube-api-access-vcdgm\") pod \"metrics-server-67dc9f7ff4-8r5zd\" (UID: \"deaaeeb5-cb5e-45fa-a284-07052b47268e\") " pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:50.898172 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:50.898146    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:54:51.170561 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:51.170488    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"]
Sep 17 10:54:51.173532 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:54:51.173511    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeaaeeb5_cb5e_45fa_a284_07052b47268e.slice/crio-180824f042c5e572271298d232643faf985734eae8e5b8a4df957a38b4e31383 WatchSource:0}: Error finding container 180824f042c5e572271298d232643faf985734eae8e5b8a4df957a38b4e31383: Status 404 returned error can't find the container with id 180824f042c5e572271298d232643faf985734eae8e5b8a4df957a38b4e31383
Sep 17 10:54:51.552715 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:51.552654    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd" event={"ID":"deaaeeb5-cb5e-45fa-a284-07052b47268e","Type":"ContainerStarted","Data":"180824f042c5e572271298d232643faf985734eae8e5b8a4df957a38b4e31383"}
Sep 17 10:54:53.563862 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:53.563823    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd" event={"ID":"deaaeeb5-cb5e-45fa-a284-07052b47268e","Type":"ContainerStarted","Data":"8d5ba89a689b8f9717de02fd5c38833f1a00f755ce833bc69854b3c016b14ef9"}
Sep 17 10:54:53.598859 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:53.598804    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd" podStartSLOduration=2.090957466 podStartE2EDuration="3.598785684s" totalImagesPullingTime="1.507828218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:54:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:54:53.596518379 +0000 UTC m=+88.912881533" watchObservedRunningTime="2026-09-17 10:54:53.598785684 +0000 UTC m=+88.915148840"
Sep 17 10:54:55.452185 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:55.452156    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:54:56.681779 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.681745    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6ff677dbd5-f74jv"]
Sep 17 10:54:56.685341 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.685317    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.689221 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.689202    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 10:54:56.689398 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.689383    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 10:54:56.689981 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.689963    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 10:54:56.690105 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.690088    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 10:54:56.690162 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.690113    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qxbsw\""
Sep 17 10:54:56.690216 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.690113    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 10:54:56.690216 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.690166    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 10:54:56.690329 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.690315    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 10:54:56.698516 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.698498    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 17 10:54:56.703357 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.703337    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6ff677dbd5-f74jv"]
Sep 17 10:54:56.848139 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.848107    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-trusted-ca-bundle\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.848139 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.848137    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-oauth-serving-cert\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.848321 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.848172    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-service-ca\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.848321 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.848188    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-config\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.848321 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.848255    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcfwk\" (UniqueName: \"kubernetes.io/projected/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-kube-api-access-wcfwk\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.848321 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.848311    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-oauth-config\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.848436 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.848354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-serving-cert\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.949686 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.949634    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcfwk\" (UniqueName: \"kubernetes.io/projected/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-kube-api-access-wcfwk\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.949686 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.949663    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-oauth-config\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.949825 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.949684    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-serving-cert\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.949825 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.949725    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-trusted-ca-bundle\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.949825 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.949752    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-oauth-serving-cert\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.949825 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.949786    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-service-ca\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.950090 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.949862    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-config\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.950571 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.950512    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-service-ca\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.950661 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.950602    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-config\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.950718 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.950659    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-oauth-serving-cert\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.950778 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.950729    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-trusted-ca-bundle\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.952251 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.952223    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-oauth-config\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.952391 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.952375    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-serving-cert\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.957917 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.957883    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcfwk\" (UniqueName: \"kubernetes.io/projected/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-kube-api-access-wcfwk\") pod \"console-6ff677dbd5-f74jv\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") " pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:56.994911 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:56.994879    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:54:57.274025 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:57.274001    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6ff677dbd5-f74jv"]
Sep 17 10:54:57.278171 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:54:57.278142    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6b785f7_7ad4_4b6d_a64b_5a4a8f0666af.slice/crio-fa64560c8b86c0286355fda67d6fc5c7720791f1a9f52043819d4b422db0b0b9 WatchSource:0}: Error finding container fa64560c8b86c0286355fda67d6fc5c7720791f1a9f52043819d4b422db0b0b9: Status 404 returned error can't find the container with id fa64560c8b86c0286355fda67d6fc5c7720791f1a9f52043819d4b422db0b0b9
Sep 17 10:54:57.580148 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:54:57.580077    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff677dbd5-f74jv" event={"ID":"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af","Type":"ContainerStarted","Data":"fa64560c8b86c0286355fda67d6fc5c7720791f1a9f52043819d4b422db0b0b9"}
Sep 17 10:55:00.466234 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.466193    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv" podUID="3d4e645b-517d-42e1-92d2-107039f0fce9" containerName="registry" containerID="cri-o://277e62bc04b348da1effcd59d751cbaed9bb637c3507de40ae75fb24c15e2565" gracePeriod=30
Sep 17 10:55:00.483711 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.483692    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-86d9d67866-242j2"
Sep 17 10:55:00.594531 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.594502    2564 generic.go:356] "Generic (PLEG): container finished" podID="3d4e645b-517d-42e1-92d2-107039f0fce9" containerID="277e62bc04b348da1effcd59d751cbaed9bb637c3507de40ae75fb24c15e2565" exitCode=0
Sep 17 10:55:00.594679 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.594590    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv" event={"ID":"3d4e645b-517d-42e1-92d2-107039f0fce9","Type":"ContainerDied","Data":"277e62bc04b348da1effcd59d751cbaed9bb637c3507de40ae75fb24c15e2565"}
Sep 17 10:55:00.596197 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.596162    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff677dbd5-f74jv" event={"ID":"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af","Type":"ContainerStarted","Data":"022066fb307304cfa6429796eef6825dc4b72bea7eaa4c4a3abf7c1d236cb0c4"}
Sep 17 10:55:00.624553 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.624508    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6ff677dbd5-f74jv" podStartSLOduration=2.041538234 podStartE2EDuration="4.624492574s" totalImagesPullingTime="2.58295434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:54:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:55:00.620806184 +0000 UTC m=+95.937169376" watchObservedRunningTime="2026-09-17 10:55:00.624492574 +0000 UTC m=+95.940855730"
Sep 17 10:55:00.701170 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.701148    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:55:00.781344 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.781316    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3d4e645b-517d-42e1-92d2-107039f0fce9-ca-trust-extracted\") pod \"3d4e645b-517d-42e1-92d2-107039f0fce9\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") "
Sep 17 10:55:00.781463 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.781374    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3d4e645b-517d-42e1-92d2-107039f0fce9-installation-pull-secrets\") pod \"3d4e645b-517d-42e1-92d2-107039f0fce9\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") "
Sep 17 10:55:00.781463 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.781411    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-bound-sa-token\") pod \"3d4e645b-517d-42e1-92d2-107039f0fce9\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") "
Sep 17 10:55:00.781463 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.781447    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3d4e645b-517d-42e1-92d2-107039f0fce9-image-registry-private-configuration\") pod \"3d4e645b-517d-42e1-92d2-107039f0fce9\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") "
Sep 17 10:55:00.781628 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.781478    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3d4e645b-517d-42e1-92d2-107039f0fce9-trusted-ca\") pod \"3d4e645b-517d-42e1-92d2-107039f0fce9\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") "
Sep 17 10:55:00.781628 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.781510    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-certificates\") pod \"3d4e645b-517d-42e1-92d2-107039f0fce9\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") "
Sep 17 10:55:00.781628 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.781571    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wlbb\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-kube-api-access-5wlbb\") pod \"3d4e645b-517d-42e1-92d2-107039f0fce9\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") "
Sep 17 10:55:00.781867 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.781835    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls\") pod \"3d4e645b-517d-42e1-92d2-107039f0fce9\" (UID: \"3d4e645b-517d-42e1-92d2-107039f0fce9\") "
Sep 17 10:55:00.783857 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.782689    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "3d4e645b-517d-42e1-92d2-107039f0fce9" (UID: "3d4e645b-517d-42e1-92d2-107039f0fce9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 10:55:00.783857 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.782960    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d4e645b-517d-42e1-92d2-107039f0fce9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "3d4e645b-517d-42e1-92d2-107039f0fce9" (UID: "3d4e645b-517d-42e1-92d2-107039f0fce9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 10:55:00.789105 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.789053    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d4e645b-517d-42e1-92d2-107039f0fce9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "3d4e645b-517d-42e1-92d2-107039f0fce9" (UID: "3d4e645b-517d-42e1-92d2-107039f0fce9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 10:55:00.789634 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.789608    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "3d4e645b-517d-42e1-92d2-107039f0fce9" (UID: "3d4e645b-517d-42e1-92d2-107039f0fce9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 10:55:00.789726 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.789626    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d4e645b-517d-42e1-92d2-107039f0fce9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "3d4e645b-517d-42e1-92d2-107039f0fce9" (UID: "3d4e645b-517d-42e1-92d2-107039f0fce9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 10:55:00.789726 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.789672    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-kube-api-access-5wlbb" (OuterVolumeSpecName: "kube-api-access-5wlbb") pod "3d4e645b-517d-42e1-92d2-107039f0fce9" (UID: "3d4e645b-517d-42e1-92d2-107039f0fce9"). InnerVolumeSpecName "kube-api-access-5wlbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 10:55:00.789797 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.789730    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3d4e645b-517d-42e1-92d2-107039f0fce9" (UID: "3d4e645b-517d-42e1-92d2-107039f0fce9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 10:55:00.794048 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.794026    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d4e645b-517d-42e1-92d2-107039f0fce9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "3d4e645b-517d-42e1-92d2-107039f0fce9" (UID: "3d4e645b-517d-42e1-92d2-107039f0fce9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:55:00.883065 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.883047    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-tls\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:55:00.883126 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.883068    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3d4e645b-517d-42e1-92d2-107039f0fce9-ca-trust-extracted\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:55:00.883126 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.883077    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3d4e645b-517d-42e1-92d2-107039f0fce9-installation-pull-secrets\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:55:00.883126 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.883086    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-bound-sa-token\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:55:00.883126 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.883096    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3d4e645b-517d-42e1-92d2-107039f0fce9-image-registry-private-configuration\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:55:00.883126 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.883104    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3d4e645b-517d-42e1-92d2-107039f0fce9-trusted-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:55:00.883126 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.883112    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3d4e645b-517d-42e1-92d2-107039f0fce9-registry-certificates\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:55:00.883126 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:00.883119    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wlbb\" (UniqueName: \"kubernetes.io/projected/3d4e645b-517d-42e1-92d2-107039f0fce9-kube-api-access-5wlbb\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:55:01.601067 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:01.601029    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv" event={"ID":"3d4e645b-517d-42e1-92d2-107039f0fce9","Type":"ContainerDied","Data":"2ad8f8f6e0c82d5fd4dafed5a25907576ee05c0a25adc5fb1bb1893ba90f6315"}
Sep 17 10:55:01.601486 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:01.601072    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fb85b79c5-rlhvv"
Sep 17 10:55:01.601486 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:01.601091    2564 scope.go:117] "RemoveContainer" containerID="277e62bc04b348da1effcd59d751cbaed9bb637c3507de40ae75fb24c15e2565"
Sep 17 10:55:01.626721 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:01.626697    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7fb85b79c5-rlhvv"]
Sep 17 10:55:01.632703 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:01.632685    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7fb85b79c5-rlhvv"]
Sep 17 10:55:03.142060 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:03.142028    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d4e645b-517d-42e1-92d2-107039f0fce9" path="/var/lib/kubelet/pods/3d4e645b-517d-42e1-92d2-107039f0fce9/volumes"
Sep 17 10:55:06.996051 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:06.996018    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:55:06.996512 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:06.996087    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:55:07.000968 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.000947    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:55:07.527113 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.527079    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79ddc584d9-zmcxb"]
Sep 17 10:55:07.527356 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.527344    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d4e645b-517d-42e1-92d2-107039f0fce9" containerName="registry"
Sep 17 10:55:07.527393 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.527357    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d4e645b-517d-42e1-92d2-107039f0fce9" containerName="registry"
Sep 17 10:55:07.527437 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.527403    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d4e645b-517d-42e1-92d2-107039f0fce9" containerName="registry"
Sep 17 10:55:07.573256 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.573224    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79ddc584d9-zmcxb"]
Sep 17 10:55:07.573411 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.573325    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.627340 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.627316    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:55:07.730610 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.730247    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlv46\" (UniqueName: \"kubernetes.io/projected/e2f24f00-8b99-4032-81c7-fcb686147480-kube-api-access-vlv46\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.730610 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.730292    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e2f24f00-8b99-4032-81c7-fcb686147480-console-oauth-config\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.730610 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.730340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-service-ca\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.730610 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.730365    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-console-config\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.730610 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.730412    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e2f24f00-8b99-4032-81c7-fcb686147480-console-serving-cert\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.730610 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.730435    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-trusted-ca-bundle\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.730610 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.730558    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-oauth-serving-cert\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.831580 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.831499    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-service-ca\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.831580 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.831534    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-console-config\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.831580 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.831572    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e2f24f00-8b99-4032-81c7-fcb686147480-console-serving-cert\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.831787 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.831594    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-trusted-ca-bundle\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.831787 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.831682    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-oauth-serving-cert\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.831787 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.831728    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlv46\" (UniqueName: \"kubernetes.io/projected/e2f24f00-8b99-4032-81c7-fcb686147480-kube-api-access-vlv46\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.831787 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.831757    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e2f24f00-8b99-4032-81c7-fcb686147480-console-oauth-config\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.832513 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.832426    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-service-ca\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.832513 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.832477    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-console-config\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.832781 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.832568    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-trusted-ca-bundle\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.832781 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.832642    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-oauth-serving-cert\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.834351 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.834320    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e2f24f00-8b99-4032-81c7-fcb686147480-console-oauth-config\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.834351 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.834340    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e2f24f00-8b99-4032-81c7-fcb686147480-console-serving-cert\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.842380 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.842358    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlv46\" (UniqueName: \"kubernetes.io/projected/e2f24f00-8b99-4032-81c7-fcb686147480-kube-api-access-vlv46\") pod \"console-79ddc584d9-zmcxb\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") " pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:07.882376 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:07.882353    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:08.185119 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:08.185042    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79ddc584d9-zmcxb"]
Sep 17 10:55:08.187823 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:55:08.187795    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2f24f00_8b99_4032_81c7_fcb686147480.slice/crio-8ac4e8fe7075e310ddfe1a7c26e618ce44d85f40247075ab1506d5959faaa71b WatchSource:0}: Error finding container 8ac4e8fe7075e310ddfe1a7c26e618ce44d85f40247075ab1506d5959faaa71b: Status 404 returned error can't find the container with id 8ac4e8fe7075e310ddfe1a7c26e618ce44d85f40247075ab1506d5959faaa71b
Sep 17 10:55:08.472173 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:08.472104    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fnw2s"
Sep 17 10:55:08.629183 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:08.629150    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79ddc584d9-zmcxb" event={"ID":"e2f24f00-8b99-4032-81c7-fcb686147480","Type":"ContainerStarted","Data":"e8cef6e80138377b1fedfa945293ba0f102080819715a176fa195d6051e92a9d"}
Sep 17 10:55:08.629422 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:08.629204    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79ddc584d9-zmcxb" event={"ID":"e2f24f00-8b99-4032-81c7-fcb686147480","Type":"ContainerStarted","Data":"8ac4e8fe7075e310ddfe1a7c26e618ce44d85f40247075ab1506d5959faaa71b"}
Sep 17 10:55:08.662313 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:08.662259    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79ddc584d9-zmcxb" podStartSLOduration=1.662241378 podStartE2EDuration="1.662241378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:55:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:55:08.660459497 +0000 UTC m=+103.976822651" watchObservedRunningTime="2026-09-17 10:55:08.662241378 +0000 UTC m=+103.978604532"
Sep 17 10:55:10.899059 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:10.899022    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:55:10.899458 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:10.899102    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:55:17.882872 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:17.882837    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:17.883280 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:17.882915    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:17.887129 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:17.887108    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:18.666522 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:18.666494    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:55:18.716286 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:18.716257    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6ff677dbd5-f74jv"]
Sep 17 10:55:30.903181 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:30.903136    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:55:30.911071 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:30.911046    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-67dc9f7ff4-8r5zd"
Sep 17 10:55:43.735523 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:43.735452    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6ff677dbd5-f74jv" podUID="a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af" containerName="console" containerID="cri-o://022066fb307304cfa6429796eef6825dc4b72bea7eaa4c4a3abf7c1d236cb0c4" gracePeriod=15
Sep 17 10:55:43.978591 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:43.978568    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:55:44.100858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.100781    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-service-ca\") pod \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") "
Sep 17 10:55:44.100858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.100828    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-config\") pod \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") "
Sep 17 10:55:44.100858 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.100859    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-trusted-ca-bundle\") pod \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") "
Sep 17 10:55:44.101157 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.100875    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-oauth-serving-cert\") pod \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") "
Sep 17 10:55:44.101157 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.100921    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcfwk\" (UniqueName: \"kubernetes.io/projected/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-kube-api-access-wcfwk\") pod \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") "
Sep 17 10:55:44.101157 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.100984    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-serving-cert\") pod \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") "
Sep 17 10:55:44.101157 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.101045    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-oauth-config\") pod \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\" (UID: \"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af\") "
Sep 17 10:55:44.101355 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.101294    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-service-ca" (OuterVolumeSpecName: "service-ca") pod "a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af" (UID: "a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 10:55:44.101355 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.101338    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af" (UID: "a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 10:55:44.101441 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.101351    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af" (UID: "a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 10:55:44.101441 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.101364    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-config" (OuterVolumeSpecName: "console-config") pod "a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af" (UID: "a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 10:55:44.103458 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.103428    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af" (UID: "a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 10:55:44.103539 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.103494    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af" (UID: "a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 10:55:44.103578 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.103536    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-kube-api-access-wcfwk" (OuterVolumeSpecName: "kube-api-access-wcfwk") pod "a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af" (UID: "a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af"). InnerVolumeSpecName "kube-api-access-wcfwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 10:55:44.201995 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.201969    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-config\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:55:44.201995 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.201990    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-trusted-ca-bundle\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:55:44.201995 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.201999    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-oauth-serving-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:55:44.202163 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.202008    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcfwk\" (UniqueName: \"kubernetes.io/projected/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-kube-api-access-wcfwk\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:55:44.202163 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.202017    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-serving-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:55:44.202163 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.202025    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-console-oauth-config\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:55:44.202163 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.202033    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:55:44.760543 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.760506    2564 generic.go:356] "Generic (PLEG): container finished" podID="a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af" containerID="022066fb307304cfa6429796eef6825dc4b72bea7eaa4c4a3abf7c1d236cb0c4" exitCode=0
Sep 17 10:55:44.760914 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.760557    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff677dbd5-f74jv"
Sep 17 10:55:44.760914 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.760579    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff677dbd5-f74jv" event={"ID":"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af","Type":"ContainerDied","Data":"022066fb307304cfa6429796eef6825dc4b72bea7eaa4c4a3abf7c1d236cb0c4"}
Sep 17 10:55:44.760914 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.760629    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff677dbd5-f74jv" event={"ID":"a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af","Type":"ContainerDied","Data":"fa64560c8b86c0286355fda67d6fc5c7720791f1a9f52043819d4b422db0b0b9"}
Sep 17 10:55:44.760914 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.760657    2564 scope.go:117] "RemoveContainer" containerID="022066fb307304cfa6429796eef6825dc4b72bea7eaa4c4a3abf7c1d236cb0c4"
Sep 17 10:55:44.769108 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.769090    2564 scope.go:117] "RemoveContainer" containerID="022066fb307304cfa6429796eef6825dc4b72bea7eaa4c4a3abf7c1d236cb0c4"
Sep 17 10:55:44.769379 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:55:44.769359    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"022066fb307304cfa6429796eef6825dc4b72bea7eaa4c4a3abf7c1d236cb0c4\": container with ID starting with 022066fb307304cfa6429796eef6825dc4b72bea7eaa4c4a3abf7c1d236cb0c4 not found: ID does not exist" containerID="022066fb307304cfa6429796eef6825dc4b72bea7eaa4c4a3abf7c1d236cb0c4"
Sep 17 10:55:44.769432 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.769388    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"022066fb307304cfa6429796eef6825dc4b72bea7eaa4c4a3abf7c1d236cb0c4"} err="failed to get container status \"022066fb307304cfa6429796eef6825dc4b72bea7eaa4c4a3abf7c1d236cb0c4\": rpc error: code = NotFound desc = could not find container \"022066fb307304cfa6429796eef6825dc4b72bea7eaa4c4a3abf7c1d236cb0c4\": container with ID starting with 022066fb307304cfa6429796eef6825dc4b72bea7eaa4c4a3abf7c1d236cb0c4 not found: ID does not exist"
Sep 17 10:55:44.793564 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.793539    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6ff677dbd5-f74jv"]
Sep 17 10:55:44.802371 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:44.802353    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6ff677dbd5-f74jv"]
Sep 17 10:55:45.144522 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:55:45.142217    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af" path="/var/lib/kubelet/pods/a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af/volumes"
Sep 17 10:56:09.600037 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.600001    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d4c6d598d-wh4dm"]
Sep 17 10:56:09.600391 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.600260    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af" containerName="console"
Sep 17 10:56:09.600391 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.600270    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af" containerName="console"
Sep 17 10:56:09.600391 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.600323    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6b785f7-7ad4-4b6d-a64b-5a4a8f0666af" containerName="console"
Sep 17 10:56:09.604184 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.604166    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.682354 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.682330    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c0bf612-05ae-400f-9511-af776c601549-console-oauth-config\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.682490 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.682362    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-trusted-ca-bundle\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.682490 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.682397    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw7zr\" (UniqueName: \"kubernetes.io/projected/1c0bf612-05ae-400f-9511-af776c601549-kube-api-access-dw7zr\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.682490 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.682466    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-service-ca\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.682611 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.682491    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-oauth-serving-cert\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.682611 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.682509    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-console-config\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.682611 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.682533    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c0bf612-05ae-400f-9511-af776c601549-console-serving-cert\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.701405 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.701382    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d4c6d598d-wh4dm"]
Sep 17 10:56:09.783394 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.783360    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw7zr\" (UniqueName: \"kubernetes.io/projected/1c0bf612-05ae-400f-9511-af776c601549-kube-api-access-dw7zr\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.783540 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.783406    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-service-ca\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.783540 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.783429    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-oauth-serving-cert\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.783540 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.783450    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-console-config\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.783540 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.783485    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c0bf612-05ae-400f-9511-af776c601549-console-serving-cert\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.783540 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.783531    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c0bf612-05ae-400f-9511-af776c601549-console-oauth-config\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.783787 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.783569    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-trusted-ca-bundle\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.784278 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.784216    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-oauth-serving-cert\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.784278 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.784263    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-service-ca\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.784546 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.784389    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-trusted-ca-bundle\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.784546 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.784500    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-console-config\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.786240 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.786220    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c0bf612-05ae-400f-9511-af776c601549-console-serving-cert\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.786329 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.786255    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c0bf612-05ae-400f-9511-af776c601549-console-oauth-config\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.790995 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.790978    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw7zr\" (UniqueName: \"kubernetes.io/projected/1c0bf612-05ae-400f-9511-af776c601549-kube-api-access-dw7zr\") pod \"console-5d4c6d598d-wh4dm\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") " pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:09.913363 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:09.913312    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:10.187072 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:10.186998    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d4c6d598d-wh4dm"]
Sep 17 10:56:10.190153 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:56:10.190120    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c0bf612_05ae_400f_9511_af776c601549.slice/crio-bd9c7533f0172c901e7813e667933a027a1016dba9cc3a21f5a888a3d71f9b32 WatchSource:0}: Error finding container bd9c7533f0172c901e7813e667933a027a1016dba9cc3a21f5a888a3d71f9b32: Status 404 returned error can't find the container with id bd9c7533f0172c901e7813e667933a027a1016dba9cc3a21f5a888a3d71f9b32
Sep 17 10:56:10.852238 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:10.852201    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d4c6d598d-wh4dm" event={"ID":"1c0bf612-05ae-400f-9511-af776c601549","Type":"ContainerStarted","Data":"cd4156e9ba5b4bedfdfcb9d46bbbc818ffe21d93911bd62e56442c5198346ac6"}
Sep 17 10:56:10.852699 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:10.852293    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d4c6d598d-wh4dm" event={"ID":"1c0bf612-05ae-400f-9511-af776c601549","Type":"ContainerStarted","Data":"bd9c7533f0172c901e7813e667933a027a1016dba9cc3a21f5a888a3d71f9b32"}
Sep 17 10:56:10.870813 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:10.870452    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d4c6d598d-wh4dm" podStartSLOduration=1.870434186 podStartE2EDuration="1.870434186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:56:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:56:10.86978885 +0000 UTC m=+166.186152004" watchObservedRunningTime="2026-09-17 10:56:10.870434186 +0000 UTC m=+166.186797339"
Sep 17 10:56:19.913872 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:19.913790    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:19.913872 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:19.913840    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:19.918230 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:19.918210    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:20.891578 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:20.891552    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 10:56:20.953368 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:20.953336    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79ddc584d9-zmcxb"]
Sep 17 10:56:45.972766 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:45.972707    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-79ddc584d9-zmcxb" podUID="e2f24f00-8b99-4032-81c7-fcb686147480" containerName="console" containerID="cri-o://e8cef6e80138377b1fedfa945293ba0f102080819715a176fa195d6051e92a9d" gracePeriod=15
Sep 17 10:56:46.213450 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.213427    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:56:46.353336 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.353307    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e2f24f00-8b99-4032-81c7-fcb686147480-console-oauth-config\") pod \"e2f24f00-8b99-4032-81c7-fcb686147480\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") "
Sep 17 10:56:46.353336 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.353338    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlv46\" (UniqueName: \"kubernetes.io/projected/e2f24f00-8b99-4032-81c7-fcb686147480-kube-api-access-vlv46\") pod \"e2f24f00-8b99-4032-81c7-fcb686147480\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") "
Sep 17 10:56:46.353513 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.353361    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-console-config\") pod \"e2f24f00-8b99-4032-81c7-fcb686147480\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") "
Sep 17 10:56:46.353513 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.353420    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e2f24f00-8b99-4032-81c7-fcb686147480-console-serving-cert\") pod \"e2f24f00-8b99-4032-81c7-fcb686147480\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") "
Sep 17 10:56:46.353513 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.353448    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-service-ca\") pod \"e2f24f00-8b99-4032-81c7-fcb686147480\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") "
Sep 17 10:56:46.353513 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.353475    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-oauth-serving-cert\") pod \"e2f24f00-8b99-4032-81c7-fcb686147480\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") "
Sep 17 10:56:46.353704 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.353527    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-trusted-ca-bundle\") pod \"e2f24f00-8b99-4032-81c7-fcb686147480\" (UID: \"e2f24f00-8b99-4032-81c7-fcb686147480\") "
Sep 17 10:56:46.353861 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.353829    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-console-config" (OuterVolumeSpecName: "console-config") pod "e2f24f00-8b99-4032-81c7-fcb686147480" (UID: "e2f24f00-8b99-4032-81c7-fcb686147480"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 10:56:46.354009 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.353852    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-service-ca" (OuterVolumeSpecName: "service-ca") pod "e2f24f00-8b99-4032-81c7-fcb686147480" (UID: "e2f24f00-8b99-4032-81c7-fcb686147480"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 10:56:46.354109 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.354085    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e2f24f00-8b99-4032-81c7-fcb686147480" (UID: "e2f24f00-8b99-4032-81c7-fcb686147480"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 10:56:46.354176 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.354149    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e2f24f00-8b99-4032-81c7-fcb686147480" (UID: "e2f24f00-8b99-4032-81c7-fcb686147480"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 10:56:46.355695 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.355671    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2f24f00-8b99-4032-81c7-fcb686147480-kube-api-access-vlv46" (OuterVolumeSpecName: "kube-api-access-vlv46") pod "e2f24f00-8b99-4032-81c7-fcb686147480" (UID: "e2f24f00-8b99-4032-81c7-fcb686147480"). InnerVolumeSpecName "kube-api-access-vlv46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 10:56:46.356183 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.356163    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2f24f00-8b99-4032-81c7-fcb686147480-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e2f24f00-8b99-4032-81c7-fcb686147480" (UID: "e2f24f00-8b99-4032-81c7-fcb686147480"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 10:56:46.356251 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.356185    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2f24f00-8b99-4032-81c7-fcb686147480-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e2f24f00-8b99-4032-81c7-fcb686147480" (UID: "e2f24f00-8b99-4032-81c7-fcb686147480"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 10:56:46.454695 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.454661    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e2f24f00-8b99-4032-81c7-fcb686147480-console-serving-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:56:46.454695 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.454694    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:56:46.454695 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.454704    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-oauth-serving-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:56:46.454856 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.454712    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-trusted-ca-bundle\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:56:46.454856 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.454721    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e2f24f00-8b99-4032-81c7-fcb686147480-console-oauth-config\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:56:46.454856 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.454729    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlv46\" (UniqueName: \"kubernetes.io/projected/e2f24f00-8b99-4032-81c7-fcb686147480-kube-api-access-vlv46\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:56:46.454856 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.454739    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e2f24f00-8b99-4032-81c7-fcb686147480-console-config\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:56:46.983423 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.983385    2564 generic.go:356] "Generic (PLEG): container finished" podID="e2f24f00-8b99-4032-81c7-fcb686147480" containerID="e8cef6e80138377b1fedfa945293ba0f102080819715a176fa195d6051e92a9d" exitCode=0
Sep 17 10:56:46.983778 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.983433    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79ddc584d9-zmcxb"
Sep 17 10:56:46.983778 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.983477    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79ddc584d9-zmcxb" event={"ID":"e2f24f00-8b99-4032-81c7-fcb686147480","Type":"ContainerDied","Data":"e8cef6e80138377b1fedfa945293ba0f102080819715a176fa195d6051e92a9d"}
Sep 17 10:56:46.983778 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.983549    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79ddc584d9-zmcxb" event={"ID":"e2f24f00-8b99-4032-81c7-fcb686147480","Type":"ContainerDied","Data":"8ac4e8fe7075e310ddfe1a7c26e618ce44d85f40247075ab1506d5959faaa71b"}
Sep 17 10:56:46.983778 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.983571    2564 scope.go:117] "RemoveContainer" containerID="e8cef6e80138377b1fedfa945293ba0f102080819715a176fa195d6051e92a9d"
Sep 17 10:56:46.991706 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.991693    2564 scope.go:117] "RemoveContainer" containerID="e8cef6e80138377b1fedfa945293ba0f102080819715a176fa195d6051e92a9d"
Sep 17 10:56:46.992022 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:56:46.992005    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8cef6e80138377b1fedfa945293ba0f102080819715a176fa195d6051e92a9d\": container with ID starting with e8cef6e80138377b1fedfa945293ba0f102080819715a176fa195d6051e92a9d not found: ID does not exist" containerID="e8cef6e80138377b1fedfa945293ba0f102080819715a176fa195d6051e92a9d"
Sep 17 10:56:46.992078 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:46.992031    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8cef6e80138377b1fedfa945293ba0f102080819715a176fa195d6051e92a9d"} err="failed to get container status \"e8cef6e80138377b1fedfa945293ba0f102080819715a176fa195d6051e92a9d\": rpc error: code = NotFound desc = could not find container \"e8cef6e80138377b1fedfa945293ba0f102080819715a176fa195d6051e92a9d\": container with ID starting with e8cef6e80138377b1fedfa945293ba0f102080819715a176fa195d6051e92a9d not found: ID does not exist"
Sep 17 10:56:47.009606 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:47.009564    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79ddc584d9-zmcxb"]
Sep 17 10:56:47.021492 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:47.021469    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79ddc584d9-zmcxb"]
Sep 17 10:56:47.141479 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:47.141453    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2f24f00-8b99-4032-81c7-fcb686147480" path="/var/lib/kubelet/pods/e2f24f00-8b99-4032-81c7-fcb686147480/volumes"
Sep 17 10:56:48.944160 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:48.944128    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4"]
Sep 17 10:56:48.944532 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:48.944376    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2f24f00-8b99-4032-81c7-fcb686147480" containerName="console"
Sep 17 10:56:48.944532 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:48.944387    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2f24f00-8b99-4032-81c7-fcb686147480" containerName="console"
Sep 17 10:56:48.944532 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:48.944441    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2f24f00-8b99-4032-81c7-fcb686147480" containerName="console"
Sep 17 10:56:48.948709 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:48.948693    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4"
Sep 17 10:56:48.951235 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:48.951188    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 10:56:48.951411 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:48.951393    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7z6jb\""
Sep 17 10:56:48.951707 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:48.951692    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 10:56:48.964647 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:48.964620    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4"]
Sep 17 10:56:49.072752 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:49.072724    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l28gh\" (UniqueName: \"kubernetes.io/projected/ffc46598-d033-475c-b8ed-fb5016a8659a-kube-api-access-l28gh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4\" (UID: \"ffc46598-d033-475c-b8ed-fb5016a8659a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4"
Sep 17 10:56:49.072890 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:49.072765    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffc46598-d033-475c-b8ed-fb5016a8659a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4\" (UID: \"ffc46598-d033-475c-b8ed-fb5016a8659a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4"
Sep 17 10:56:49.072890 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:49.072786    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffc46598-d033-475c-b8ed-fb5016a8659a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4\" (UID: \"ffc46598-d033-475c-b8ed-fb5016a8659a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4"
Sep 17 10:56:49.174156 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:49.174128    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffc46598-d033-475c-b8ed-fb5016a8659a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4\" (UID: \"ffc46598-d033-475c-b8ed-fb5016a8659a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4"
Sep 17 10:56:49.174265 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:49.174164    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffc46598-d033-475c-b8ed-fb5016a8659a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4\" (UID: \"ffc46598-d033-475c-b8ed-fb5016a8659a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4"
Sep 17 10:56:49.174265 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:49.174236    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l28gh\" (UniqueName: \"kubernetes.io/projected/ffc46598-d033-475c-b8ed-fb5016a8659a-kube-api-access-l28gh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4\" (UID: \"ffc46598-d033-475c-b8ed-fb5016a8659a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4"
Sep 17 10:56:49.174560 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:49.174540    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffc46598-d033-475c-b8ed-fb5016a8659a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4\" (UID: \"ffc46598-d033-475c-b8ed-fb5016a8659a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4"
Sep 17 10:56:49.174603 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:49.174554    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffc46598-d033-475c-b8ed-fb5016a8659a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4\" (UID: \"ffc46598-d033-475c-b8ed-fb5016a8659a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4"
Sep 17 10:56:49.183702 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:49.183680    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l28gh\" (UniqueName: \"kubernetes.io/projected/ffc46598-d033-475c-b8ed-fb5016a8659a-kube-api-access-l28gh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4\" (UID: \"ffc46598-d033-475c-b8ed-fb5016a8659a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4"
Sep 17 10:56:49.257829 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:49.257804    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4"
Sep 17 10:56:49.532556 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:49.532407    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4"]
Sep 17 10:56:49.535291 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:56:49.535265    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffc46598_d033_475c_b8ed_fb5016a8659a.slice/crio-2b9cf87aa4cdefe7d90eee7eb6625a4bc5251d202274344596df9e1bf401af9f WatchSource:0}: Error finding container 2b9cf87aa4cdefe7d90eee7eb6625a4bc5251d202274344596df9e1bf401af9f: Status 404 returned error can't find the container with id 2b9cf87aa4cdefe7d90eee7eb6625a4bc5251d202274344596df9e1bf401af9f
Sep 17 10:56:49.996657 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:49.996628    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4" event={"ID":"ffc46598-d033-475c-b8ed-fb5016a8659a","Type":"ContainerStarted","Data":"2b9cf87aa4cdefe7d90eee7eb6625a4bc5251d202274344596df9e1bf401af9f"}
Sep 17 10:56:57.026853 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:57.026817    2564 generic.go:356] "Generic (PLEG): container finished" podID="ffc46598-d033-475c-b8ed-fb5016a8659a" containerID="3e0f9e2bf4322f23295ee7f0a0fd357658f2625d9c49f7b6bc84176f979af037" exitCode=0
Sep 17 10:56:57.027270 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:57.026888    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4" event={"ID":"ffc46598-d033-475c-b8ed-fb5016a8659a","Type":"ContainerDied","Data":"3e0f9e2bf4322f23295ee7f0a0fd357658f2625d9c49f7b6bc84176f979af037"}
Sep 17 10:56:59.035329 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:59.035256    2564 generic.go:356] "Generic (PLEG): container finished" podID="ffc46598-d033-475c-b8ed-fb5016a8659a" containerID="2fcc99cf0bc95ec7543aac2652c5797806a1ca24978b6acfb22a70a71446e864" exitCode=0
Sep 17 10:56:59.035663 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:56:59.035334    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4" event={"ID":"ffc46598-d033-475c-b8ed-fb5016a8659a","Type":"ContainerDied","Data":"2fcc99cf0bc95ec7543aac2652c5797806a1ca24978b6acfb22a70a71446e864"}
Sep 17 10:57:06.065551 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:06.065517    2564 generic.go:356] "Generic (PLEG): container finished" podID="ffc46598-d033-475c-b8ed-fb5016a8659a" containerID="3ae19a28431e43aedcf27d23f0c3d632cb629371ea53d63951c3ea6c877eb609" exitCode=0
Sep 17 10:57:06.065959 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:06.065596    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4" event={"ID":"ffc46598-d033-475c-b8ed-fb5016a8659a","Type":"ContainerDied","Data":"3ae19a28431e43aedcf27d23f0c3d632cb629371ea53d63951c3ea6c877eb609"}
Sep 17 10:57:07.188728 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:07.188708    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4"
Sep 17 10:57:07.199096 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:07.199070    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffc46598-d033-475c-b8ed-fb5016a8659a-bundle\") pod \"ffc46598-d033-475c-b8ed-fb5016a8659a\" (UID: \"ffc46598-d033-475c-b8ed-fb5016a8659a\") "
Sep 17 10:57:07.199181 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:07.199137    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l28gh\" (UniqueName: \"kubernetes.io/projected/ffc46598-d033-475c-b8ed-fb5016a8659a-kube-api-access-l28gh\") pod \"ffc46598-d033-475c-b8ed-fb5016a8659a\" (UID: \"ffc46598-d033-475c-b8ed-fb5016a8659a\") "
Sep 17 10:57:07.199181 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:07.199163    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffc46598-d033-475c-b8ed-fb5016a8659a-util\") pod \"ffc46598-d033-475c-b8ed-fb5016a8659a\" (UID: \"ffc46598-d033-475c-b8ed-fb5016a8659a\") "
Sep 17 10:57:07.199776 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:07.199752    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc46598-d033-475c-b8ed-fb5016a8659a-bundle" (OuterVolumeSpecName: "bundle") pod "ffc46598-d033-475c-b8ed-fb5016a8659a" (UID: "ffc46598-d033-475c-b8ed-fb5016a8659a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:57:07.201351 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:07.201329    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffc46598-d033-475c-b8ed-fb5016a8659a-kube-api-access-l28gh" (OuterVolumeSpecName: "kube-api-access-l28gh") pod "ffc46598-d033-475c-b8ed-fb5016a8659a" (UID: "ffc46598-d033-475c-b8ed-fb5016a8659a"). InnerVolumeSpecName "kube-api-access-l28gh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 10:57:07.203350 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:07.203323    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc46598-d033-475c-b8ed-fb5016a8659a-util" (OuterVolumeSpecName: "util") pod "ffc46598-d033-475c-b8ed-fb5016a8659a" (UID: "ffc46598-d033-475c-b8ed-fb5016a8659a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:57:07.300617 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:07.300588    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffc46598-d033-475c-b8ed-fb5016a8659a-bundle\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:57:07.300617 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:07.300613    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l28gh\" (UniqueName: \"kubernetes.io/projected/ffc46598-d033-475c-b8ed-fb5016a8659a-kube-api-access-l28gh\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:57:07.300769 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:07.300627    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffc46598-d033-475c-b8ed-fb5016a8659a-util\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:57:08.074601 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:08.074573    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4" event={"ID":"ffc46598-d033-475c-b8ed-fb5016a8659a","Type":"ContainerDied","Data":"2b9cf87aa4cdefe7d90eee7eb6625a4bc5251d202274344596df9e1bf401af9f"}
Sep 17 10:57:08.074601 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:08.074590    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v8bx4"
Sep 17 10:57:08.074826 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:08.074635    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b9cf87aa4cdefe7d90eee7eb6625a4bc5251d202274344596df9e1bf401af9f"
Sep 17 10:57:12.708248 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.708210    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nwgmc"]
Sep 17 10:57:12.708697 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.708609    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffc46598-d033-475c-b8ed-fb5016a8659a" containerName="util"
Sep 17 10:57:12.708697 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.708626    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffc46598-d033-475c-b8ed-fb5016a8659a" containerName="util"
Sep 17 10:57:12.708697 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.708643    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffc46598-d033-475c-b8ed-fb5016a8659a" containerName="extract"
Sep 17 10:57:12.708697 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.708651    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffc46598-d033-475c-b8ed-fb5016a8659a" containerName="extract"
Sep 17 10:57:12.708697 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.708670    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffc46598-d033-475c-b8ed-fb5016a8659a" containerName="pull"
Sep 17 10:57:12.708697 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.708677    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffc46598-d033-475c-b8ed-fb5016a8659a" containerName="pull"
Sep 17 10:57:12.709090 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.708738    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffc46598-d033-475c-b8ed-fb5016a8659a" containerName="extract"
Sep 17 10:57:12.747136 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.747109    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nwgmc"]
Sep 17 10:57:12.747269 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.747196    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nwgmc"
Sep 17 10:57:12.751326 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.751307    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 17 10:57:12.752165 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.752016    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-s6xjp\""
Sep 17 10:57:12.752406 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.752390    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 17 10:57:12.842687 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.842661    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz5t6\" (UniqueName: \"kubernetes.io/projected/7c8aae53-1a45-4ced-b43a-6abb20fc3503-kube-api-access-jz5t6\") pod \"cert-manager-operator-controller-manager-55c444bf86-nwgmc\" (UID: \"7c8aae53-1a45-4ced-b43a-6abb20fc3503\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nwgmc"
Sep 17 10:57:12.842830 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.842725    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c8aae53-1a45-4ced-b43a-6abb20fc3503-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-nwgmc\" (UID: \"7c8aae53-1a45-4ced-b43a-6abb20fc3503\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nwgmc"
Sep 17 10:57:12.944061 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.944032    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c8aae53-1a45-4ced-b43a-6abb20fc3503-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-nwgmc\" (UID: \"7c8aae53-1a45-4ced-b43a-6abb20fc3503\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nwgmc"
Sep 17 10:57:12.944178 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.944079    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz5t6\" (UniqueName: \"kubernetes.io/projected/7c8aae53-1a45-4ced-b43a-6abb20fc3503-kube-api-access-jz5t6\") pod \"cert-manager-operator-controller-manager-55c444bf86-nwgmc\" (UID: \"7c8aae53-1a45-4ced-b43a-6abb20fc3503\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nwgmc"
Sep 17 10:57:12.944396 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.944379    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c8aae53-1a45-4ced-b43a-6abb20fc3503-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-nwgmc\" (UID: \"7c8aae53-1a45-4ced-b43a-6abb20fc3503\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nwgmc"
Sep 17 10:57:12.960096 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:12.960047    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz5t6\" (UniqueName: \"kubernetes.io/projected/7c8aae53-1a45-4ced-b43a-6abb20fc3503-kube-api-access-jz5t6\") pod \"cert-manager-operator-controller-manager-55c444bf86-nwgmc\" (UID: \"7c8aae53-1a45-4ced-b43a-6abb20fc3503\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nwgmc"
Sep 17 10:57:13.060907 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:13.060877    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nwgmc"
Sep 17 10:57:13.381757 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:13.381701    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nwgmc"]
Sep 17 10:57:13.384132 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:57:13.384103    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c8aae53_1a45_4ced_b43a_6abb20fc3503.slice/crio-536511c96769c67b196db79311a51f56f2f846dec4905e911367663216d6d920 WatchSource:0}: Error finding container 536511c96769c67b196db79311a51f56f2f846dec4905e911367663216d6d920: Status 404 returned error can't find the container with id 536511c96769c67b196db79311a51f56f2f846dec4905e911367663216d6d920
Sep 17 10:57:14.097542 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:14.097512    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nwgmc" event={"ID":"7c8aae53-1a45-4ced-b43a-6abb20fc3503","Type":"ContainerStarted","Data":"536511c96769c67b196db79311a51f56f2f846dec4905e911367663216d6d920"}
Sep 17 10:57:15.103620 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:15.103590    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nwgmc" event={"ID":"7c8aae53-1a45-4ced-b43a-6abb20fc3503","Type":"ContainerStarted","Data":"015bb11e2fe807ed7ade40abc56f74db80a48ad7f05fca64fd32604d37e28e55"}
Sep 17 10:57:15.135436 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:15.135385    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-nwgmc" podStartSLOduration=1.5042463700000002 podStartE2EDuration="3.135362957s" totalImagesPullingTime="1.631116587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:57:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:57:15.1341362 +0000 UTC m=+230.450499368" watchObservedRunningTime="2026-09-17 10:57:15.135362957 +0000 UTC m=+230.451726113"
Sep 17 10:57:16.268987 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.268953    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf"]
Sep 17 10:57:16.271180 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.271164    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf"
Sep 17 10:57:16.274749 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.274730    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 10:57:16.275113 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.275094    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7z6jb\""
Sep 17 10:57:16.275207 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.275094    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 10:57:16.291408 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.291383    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf"]
Sep 17 10:57:16.369514 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.369485    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmdbx\" (UniqueName: \"kubernetes.io/projected/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-kube-api-access-tmdbx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf\" (UID: \"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf"
Sep 17 10:57:16.369610 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.369533    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf\" (UID: \"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf"
Sep 17 10:57:16.369610 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.369563    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf\" (UID: \"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf"
Sep 17 10:57:16.469962 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.469939    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf\" (UID: \"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf"
Sep 17 10:57:16.470070 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.469970    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf\" (UID: \"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf"
Sep 17 10:57:16.470070 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.470007    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmdbx\" (UniqueName: \"kubernetes.io/projected/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-kube-api-access-tmdbx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf\" (UID: \"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf"
Sep 17 10:57:16.470378 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.470356    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf\" (UID: \"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf"
Sep 17 10:57:16.470424 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.470366    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf\" (UID: \"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf"
Sep 17 10:57:16.482405 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.482377    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmdbx\" (UniqueName: \"kubernetes.io/projected/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-kube-api-access-tmdbx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf\" (UID: \"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf"
Sep 17 10:57:16.580303 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.580241    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf"
Sep 17 10:57:16.858120 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:16.858095    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf"]
Sep 17 10:57:16.860731 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:57:16.860705    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4c5628a_8aa4_4459_8eb0_c8b18a0bbc38.slice/crio-1e2834ad19ce9393a739cbf4914b63635eac91b3dabfa7b48c0c4669ef69c922 WatchSource:0}: Error finding container 1e2834ad19ce9393a739cbf4914b63635eac91b3dabfa7b48c0c4669ef69c922: Status 404 returned error can't find the container with id 1e2834ad19ce9393a739cbf4914b63635eac91b3dabfa7b48c0c4669ef69c922
Sep 17 10:57:17.113512 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:17.113433    2564 generic.go:356] "Generic (PLEG): container finished" podID="f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38" containerID="5dae7bc54a38da639fad109b6e201e7b2ed2c043032e80df9d09331846b53882" exitCode=0
Sep 17 10:57:17.113630 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:17.113523    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf" event={"ID":"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38","Type":"ContainerDied","Data":"5dae7bc54a38da639fad109b6e201e7b2ed2c043032e80df9d09331846b53882"}
Sep 17 10:57:17.113630 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:17.113563    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf" event={"ID":"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38","Type":"ContainerStarted","Data":"1e2834ad19ce9393a739cbf4914b63635eac91b3dabfa7b48c0c4669ef69c922"}
Sep 17 10:57:17.726119 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:17.726089    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-25qqw"]
Sep 17 10:57:17.728239 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:17.728223    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-25qqw"
Sep 17 10:57:17.731527 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:17.731508    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 17 10:57:17.731660 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:17.731555    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 17 10:57:17.731660 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:17.731575    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-9b4fz\""
Sep 17 10:57:17.743535 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:17.743476    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-25qqw"]
Sep 17 10:57:17.780757 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:17.780731    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08b5ac2f-cd5d-4f26-9a13-382fad06f880-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-25qqw\" (UID: \"08b5ac2f-cd5d-4f26-9a13-382fad06f880\") " pod="cert-manager/cert-manager-webhook-6486ffd997-25qqw"
Sep 17 10:57:17.780872 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:17.780765    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c6gq\" (UniqueName: \"kubernetes.io/projected/08b5ac2f-cd5d-4f26-9a13-382fad06f880-kube-api-access-9c6gq\") pod \"cert-manager-webhook-6486ffd997-25qqw\" (UID: \"08b5ac2f-cd5d-4f26-9a13-382fad06f880\") " pod="cert-manager/cert-manager-webhook-6486ffd997-25qqw"
Sep 17 10:57:17.881809 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:17.881779    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08b5ac2f-cd5d-4f26-9a13-382fad06f880-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-25qqw\" (UID: \"08b5ac2f-cd5d-4f26-9a13-382fad06f880\") " pod="cert-manager/cert-manager-webhook-6486ffd997-25qqw"
Sep 17 10:57:17.881992 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:17.881920    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c6gq\" (UniqueName: \"kubernetes.io/projected/08b5ac2f-cd5d-4f26-9a13-382fad06f880-kube-api-access-9c6gq\") pod \"cert-manager-webhook-6486ffd997-25qqw\" (UID: \"08b5ac2f-cd5d-4f26-9a13-382fad06f880\") " pod="cert-manager/cert-manager-webhook-6486ffd997-25qqw"
Sep 17 10:57:17.903676 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:17.903651    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c6gq\" (UniqueName: \"kubernetes.io/projected/08b5ac2f-cd5d-4f26-9a13-382fad06f880-kube-api-access-9c6gq\") pod \"cert-manager-webhook-6486ffd997-25qqw\" (UID: \"08b5ac2f-cd5d-4f26-9a13-382fad06f880\") " pod="cert-manager/cert-manager-webhook-6486ffd997-25qqw"
Sep 17 10:57:17.903817 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:17.903797    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08b5ac2f-cd5d-4f26-9a13-382fad06f880-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-25qqw\" (UID: \"08b5ac2f-cd5d-4f26-9a13-382fad06f880\") " pod="cert-manager/cert-manager-webhook-6486ffd997-25qqw"
Sep 17 10:57:18.043287 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:18.043204    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-25qqw"
Sep 17 10:57:18.327203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:18.327176    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-25qqw"]
Sep 17 10:57:18.329813 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:57:18.329789    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08b5ac2f_cd5d_4f26_9a13_382fad06f880.slice/crio-9830506c117d704d8e5db2b88fc300e9fcb03af7164efd4c41ef8e688defcf9a WatchSource:0}: Error finding container 9830506c117d704d8e5db2b88fc300e9fcb03af7164efd4c41ef8e688defcf9a: Status 404 returned error can't find the container with id 9830506c117d704d8e5db2b88fc300e9fcb03af7164efd4c41ef8e688defcf9a
Sep 17 10:57:19.125309 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:19.125275    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-25qqw" event={"ID":"08b5ac2f-cd5d-4f26-9a13-382fad06f880","Type":"ContainerStarted","Data":"9830506c117d704d8e5db2b88fc300e9fcb03af7164efd4c41ef8e688defcf9a"}
Sep 17 10:57:20.133207 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:20.133172    2564 generic.go:356] "Generic (PLEG): container finished" podID="f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38" containerID="c61054a6334c928315f6822cf4883fc5be41055fb93aba2f77def9a0f1431b0b" exitCode=0
Sep 17 10:57:20.133684 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:20.133276    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf" event={"ID":"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38","Type":"ContainerDied","Data":"c61054a6334c928315f6822cf4883fc5be41055fb93aba2f77def9a0f1431b0b"}
Sep 17 10:57:21.142063 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:21.142025    2564 generic.go:356] "Generic (PLEG): container finished" podID="f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38" containerID="11a7c0d21a321e2dfbef37e420f78c0d5cbeb54e1f61eeb4991d012e9956d0dd" exitCode=0
Sep 17 10:57:21.142488 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:21.142273    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf" event={"ID":"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38","Type":"ContainerDied","Data":"11a7c0d21a321e2dfbef37e420f78c0d5cbeb54e1f61eeb4991d012e9956d0dd"}
Sep 17 10:57:22.148217 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:22.148180    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-25qqw" event={"ID":"08b5ac2f-cd5d-4f26-9a13-382fad06f880","Type":"ContainerStarted","Data":"4ae0ee73c63a25903bdc5eb8839b486dbe7c826d3b125abc6cfab7346aae3fc4"}
Sep 17 10:57:22.172530 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:22.172476    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-25qqw" podStartSLOduration=2.150631237 podStartE2EDuration="5.172460936s" totalImagesPullingTime="3.021829699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:57:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:57:22.171347202 +0000 UTC m=+237.487710359" watchObservedRunningTime="2026-09-17 10:57:22.172460936 +0000 UTC m=+237.488824091"
Sep 17 10:57:22.271245 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:22.271224    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf"
Sep 17 10:57:22.319252 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:22.319226    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-bundle\") pod \"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38\" (UID: \"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38\") "
Sep 17 10:57:22.319355 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:22.319315    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-util\") pod \"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38\" (UID: \"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38\") "
Sep 17 10:57:22.319355 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:22.319345    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmdbx\" (UniqueName: \"kubernetes.io/projected/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-kube-api-access-tmdbx\") pod \"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38\" (UID: \"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38\") "
Sep 17 10:57:22.319626 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:22.319597    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-bundle" (OuterVolumeSpecName: "bundle") pod "f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38" (UID: "f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:57:22.322024 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:22.321812    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-kube-api-access-tmdbx" (OuterVolumeSpecName: "kube-api-access-tmdbx") pod "f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38" (UID: "f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38"). InnerVolumeSpecName "kube-api-access-tmdbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 10:57:22.326152 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:22.326119    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-util" (OuterVolumeSpecName: "util") pod "f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38" (UID: "f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:57:22.420021 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:22.419931    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-util\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:57:22.420021 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:22.419974    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmdbx\" (UniqueName: \"kubernetes.io/projected/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-kube-api-access-tmdbx\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:57:22.420021 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:22.419989    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38-bundle\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:57:23.153370 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:23.153330    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf" event={"ID":"f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38","Type":"ContainerDied","Data":"1e2834ad19ce9393a739cbf4914b63635eac91b3dabfa7b48c0c4669ef69c922"}
Sep 17 10:57:23.153370 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:23.153358    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftwnpf"
Sep 17 10:57:23.153370 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:23.153376    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e2834ad19ce9393a739cbf4914b63635eac91b3dabfa7b48c0c4669ef69c922"
Sep 17 10:57:23.153865 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:23.153437    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-25qqw"
Sep 17 10:57:28.341459 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.341424    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-x9b6f"]
Sep 17 10:57:28.341885 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.341675    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38" containerName="util"
Sep 17 10:57:28.341885 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.341685    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38" containerName="util"
Sep 17 10:57:28.341885 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.341700    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38" containerName="pull"
Sep 17 10:57:28.341885 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.341706    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38" containerName="pull"
Sep 17 10:57:28.341885 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.341717    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38" containerName="extract"
Sep 17 10:57:28.341885 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.341723    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38" containerName="extract"
Sep 17 10:57:28.341885 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.341771    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4c5628a-8aa4-4459-8eb0-c8b18a0bbc38" containerName="extract"
Sep 17 10:57:28.346186 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.346170    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-x9b6f"
Sep 17 10:57:28.348189 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.348170    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-56gkb\""
Sep 17 10:57:28.363016 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.362997    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-x9b6f"]
Sep 17 10:57:28.465305 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.465279    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ccd871e3-9c81-4e6b-85bd-98e6308692ec-bound-sa-token\") pod \"cert-manager-55b4d954d5-x9b6f\" (UID: \"ccd871e3-9c81-4e6b-85bd-98e6308692ec\") " pod="cert-manager/cert-manager-55b4d954d5-x9b6f"
Sep 17 10:57:28.465405 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.465312    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chqvr\" (UniqueName: \"kubernetes.io/projected/ccd871e3-9c81-4e6b-85bd-98e6308692ec-kube-api-access-chqvr\") pod \"cert-manager-55b4d954d5-x9b6f\" (UID: \"ccd871e3-9c81-4e6b-85bd-98e6308692ec\") " pod="cert-manager/cert-manager-55b4d954d5-x9b6f"
Sep 17 10:57:28.565981 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.565957    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chqvr\" (UniqueName: \"kubernetes.io/projected/ccd871e3-9c81-4e6b-85bd-98e6308692ec-kube-api-access-chqvr\") pod \"cert-manager-55b4d954d5-x9b6f\" (UID: \"ccd871e3-9c81-4e6b-85bd-98e6308692ec\") " pod="cert-manager/cert-manager-55b4d954d5-x9b6f"
Sep 17 10:57:28.566102 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.566038    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ccd871e3-9c81-4e6b-85bd-98e6308692ec-bound-sa-token\") pod \"cert-manager-55b4d954d5-x9b6f\" (UID: \"ccd871e3-9c81-4e6b-85bd-98e6308692ec\") " pod="cert-manager/cert-manager-55b4d954d5-x9b6f"
Sep 17 10:57:28.581549 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.581522    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ccd871e3-9c81-4e6b-85bd-98e6308692ec-bound-sa-token\") pod \"cert-manager-55b4d954d5-x9b6f\" (UID: \"ccd871e3-9c81-4e6b-85bd-98e6308692ec\") " pod="cert-manager/cert-manager-55b4d954d5-x9b6f"
Sep 17 10:57:28.581632 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.581596    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chqvr\" (UniqueName: \"kubernetes.io/projected/ccd871e3-9c81-4e6b-85bd-98e6308692ec-kube-api-access-chqvr\") pod \"cert-manager-55b4d954d5-x9b6f\" (UID: \"ccd871e3-9c81-4e6b-85bd-98e6308692ec\") " pod="cert-manager/cert-manager-55b4d954d5-x9b6f"
Sep 17 10:57:28.655203 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.655144    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-x9b6f"
Sep 17 10:57:28.935715 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:28.935650    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-x9b6f"]
Sep 17 10:57:28.938396 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:57:28.938368    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podccd871e3_9c81_4e6b_85bd_98e6308692ec.slice/crio-6c5456f35d4740967253e418cd43925647810a7ee8a420244cd02483b013b204 WatchSource:0}: Error finding container 6c5456f35d4740967253e418cd43925647810a7ee8a420244cd02483b013b204: Status 404 returned error can't find the container with id 6c5456f35d4740967253e418cd43925647810a7ee8a420244cd02483b013b204
Sep 17 10:57:29.161436 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:29.161407    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-25qqw"
Sep 17 10:57:29.183340 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:29.183309    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-x9b6f" event={"ID":"ccd871e3-9c81-4e6b-85bd-98e6308692ec","Type":"ContainerStarted","Data":"ec607267c0312749a08782ba27afdf27a4736d3657623bd311fc510b91ec4104"}
Sep 17 10:57:29.183340 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:29.183343    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-x9b6f" event={"ID":"ccd871e3-9c81-4e6b-85bd-98e6308692ec","Type":"ContainerStarted","Data":"6c5456f35d4740967253e418cd43925647810a7ee8a420244cd02483b013b204"}
Sep 17 10:57:29.252112 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:29.252069    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-x9b6f" podStartSLOduration=1.252052776 podStartE2EDuration="1.252052776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:57:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:57:29.250081342 +0000 UTC m=+244.566444498" watchObservedRunningTime="2026-09-17 10:57:29.252052776 +0000 UTC m=+244.568415931"
Sep 17 10:57:36.282825 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.282794    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5"]
Sep 17 10:57:36.286677 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.286659    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5"
Sep 17 10:57:36.289096 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.289079    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 10:57:36.289175 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.289081    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7z6jb\""
Sep 17 10:57:36.289565 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.289550    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 10:57:36.295591 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.295576    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5"]
Sep 17 10:57:36.425985 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.425942    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z6dh\" (UniqueName: \"kubernetes.io/projected/bacefdfe-6069-449d-b814-4f7a24915796-kube-api-access-6z6dh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5\" (UID: \"bacefdfe-6069-449d-b814-4f7a24915796\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5"
Sep 17 10:57:36.426121 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.425989    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bacefdfe-6069-449d-b814-4f7a24915796-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5\" (UID: \"bacefdfe-6069-449d-b814-4f7a24915796\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5"
Sep 17 10:57:36.426121 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.426104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bacefdfe-6069-449d-b814-4f7a24915796-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5\" (UID: \"bacefdfe-6069-449d-b814-4f7a24915796\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5"
Sep 17 10:57:36.527084 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.527058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bacefdfe-6069-449d-b814-4f7a24915796-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5\" (UID: \"bacefdfe-6069-449d-b814-4f7a24915796\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5"
Sep 17 10:57:36.527227 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.527100    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z6dh\" (UniqueName: \"kubernetes.io/projected/bacefdfe-6069-449d-b814-4f7a24915796-kube-api-access-6z6dh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5\" (UID: \"bacefdfe-6069-449d-b814-4f7a24915796\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5"
Sep 17 10:57:36.527227 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.527119    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bacefdfe-6069-449d-b814-4f7a24915796-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5\" (UID: \"bacefdfe-6069-449d-b814-4f7a24915796\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5"
Sep 17 10:57:36.527480 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.527460    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bacefdfe-6069-449d-b814-4f7a24915796-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5\" (UID: \"bacefdfe-6069-449d-b814-4f7a24915796\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5"
Sep 17 10:57:36.527480 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.527473    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bacefdfe-6069-449d-b814-4f7a24915796-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5\" (UID: \"bacefdfe-6069-449d-b814-4f7a24915796\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5"
Sep 17 10:57:36.535991 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.535933    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z6dh\" (UniqueName: \"kubernetes.io/projected/bacefdfe-6069-449d-b814-4f7a24915796-kube-api-access-6z6dh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5\" (UID: \"bacefdfe-6069-449d-b814-4f7a24915796\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5"
Sep 17 10:57:36.596586 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.596561    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5"
Sep 17 10:57:36.875497 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:36.875473    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5"]
Sep 17 10:57:36.877945 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:57:36.877917    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbacefdfe_6069_449d_b814_4f7a24915796.slice/crio-18ca69e8313cb2b629111fef12630ece3d87e46080a2b8f1735d6f485a99a195 WatchSource:0}: Error finding container 18ca69e8313cb2b629111fef12630ece3d87e46080a2b8f1735d6f485a99a195: Status 404 returned error can't find the container with id 18ca69e8313cb2b629111fef12630ece3d87e46080a2b8f1735d6f485a99a195
Sep 17 10:57:37.219383 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:37.219306    2564 generic.go:356] "Generic (PLEG): container finished" podID="bacefdfe-6069-449d-b814-4f7a24915796" containerID="1191390542f4ffde864bf12caed593b32ae3bb88cfbf17fdf7578b54c6a422ae" exitCode=0
Sep 17 10:57:37.219529 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:37.219382    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5" event={"ID":"bacefdfe-6069-449d-b814-4f7a24915796","Type":"ContainerDied","Data":"1191390542f4ffde864bf12caed593b32ae3bb88cfbf17fdf7578b54c6a422ae"}
Sep 17 10:57:37.219529 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:37.219418    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5" event={"ID":"bacefdfe-6069-449d-b814-4f7a24915796","Type":"ContainerStarted","Data":"18ca69e8313cb2b629111fef12630ece3d87e46080a2b8f1735d6f485a99a195"}
Sep 17 10:57:38.226290 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:38.226207    2564 generic.go:356] "Generic (PLEG): container finished" podID="bacefdfe-6069-449d-b814-4f7a24915796" containerID="b068dafdda277a8da5d1915d047b4684e82a12ad6a45d57830914a00dce9fd72" exitCode=0
Sep 17 10:57:38.226689 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:38.226286    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5" event={"ID":"bacefdfe-6069-449d-b814-4f7a24915796","Type":"ContainerDied","Data":"b068dafdda277a8da5d1915d047b4684e82a12ad6a45d57830914a00dce9fd72"}
Sep 17 10:57:39.232121 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:39.232085    2564 generic.go:356] "Generic (PLEG): container finished" podID="bacefdfe-6069-449d-b814-4f7a24915796" containerID="bf992de7ecf51a0c154de884f288967e32366b5d82928eb333933334c017fb06" exitCode=0
Sep 17 10:57:39.232500 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:39.232166    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5" event={"ID":"bacefdfe-6069-449d-b814-4f7a24915796","Type":"ContainerDied","Data":"bf992de7ecf51a0c154de884f288967e32366b5d82928eb333933334c017fb06"}
Sep 17 10:57:40.381280 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:40.381257    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5"
Sep 17 10:57:40.459315 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:40.459286    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z6dh\" (UniqueName: \"kubernetes.io/projected/bacefdfe-6069-449d-b814-4f7a24915796-kube-api-access-6z6dh\") pod \"bacefdfe-6069-449d-b814-4f7a24915796\" (UID: \"bacefdfe-6069-449d-b814-4f7a24915796\") "
Sep 17 10:57:40.459458 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:40.459342    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bacefdfe-6069-449d-b814-4f7a24915796-util\") pod \"bacefdfe-6069-449d-b814-4f7a24915796\" (UID: \"bacefdfe-6069-449d-b814-4f7a24915796\") "
Sep 17 10:57:40.459458 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:40.459390    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bacefdfe-6069-449d-b814-4f7a24915796-bundle\") pod \"bacefdfe-6069-449d-b814-4f7a24915796\" (UID: \"bacefdfe-6069-449d-b814-4f7a24915796\") "
Sep 17 10:57:40.460155 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:40.460128    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bacefdfe-6069-449d-b814-4f7a24915796-bundle" (OuterVolumeSpecName: "bundle") pod "bacefdfe-6069-449d-b814-4f7a24915796" (UID: "bacefdfe-6069-449d-b814-4f7a24915796"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:57:40.461529 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:40.461503    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bacefdfe-6069-449d-b814-4f7a24915796-kube-api-access-6z6dh" (OuterVolumeSpecName: "kube-api-access-6z6dh") pod "bacefdfe-6069-449d-b814-4f7a24915796" (UID: "bacefdfe-6069-449d-b814-4f7a24915796"). InnerVolumeSpecName "kube-api-access-6z6dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 10:57:40.465007 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:40.464976    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bacefdfe-6069-449d-b814-4f7a24915796-util" (OuterVolumeSpecName: "util") pod "bacefdfe-6069-449d-b814-4f7a24915796" (UID: "bacefdfe-6069-449d-b814-4f7a24915796"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:57:40.560733 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:40.560678    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z6dh\" (UniqueName: \"kubernetes.io/projected/bacefdfe-6069-449d-b814-4f7a24915796-kube-api-access-6z6dh\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:57:40.560733 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:40.560700    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bacefdfe-6069-449d-b814-4f7a24915796-util\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:57:40.560733 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:40.560711    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bacefdfe-6069-449d-b814-4f7a24915796-bundle\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:57:41.242600 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:41.242572    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5"
Sep 17 10:57:41.242600 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:41.242582    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsgfc5" event={"ID":"bacefdfe-6069-449d-b814-4f7a24915796","Type":"ContainerDied","Data":"18ca69e8313cb2b629111fef12630ece3d87e46080a2b8f1735d6f485a99a195"}
Sep 17 10:57:41.242778 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:41.242621    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18ca69e8313cb2b629111fef12630ece3d87e46080a2b8f1735d6f485a99a195"
Sep 17 10:57:47.540438 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.540361    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9"]
Sep 17 10:57:47.540825 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.540622    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bacefdfe-6069-449d-b814-4f7a24915796" containerName="pull"
Sep 17 10:57:47.540825 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.540634    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bacefdfe-6069-449d-b814-4f7a24915796" containerName="pull"
Sep 17 10:57:47.540825 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.540645    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bacefdfe-6069-449d-b814-4f7a24915796" containerName="util"
Sep 17 10:57:47.540825 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.540650    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bacefdfe-6069-449d-b814-4f7a24915796" containerName="util"
Sep 17 10:57:47.540825 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.540662    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bacefdfe-6069-449d-b814-4f7a24915796" containerName="extract"
Sep 17 10:57:47.540825 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.540668    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bacefdfe-6069-449d-b814-4f7a24915796" containerName="extract"
Sep 17 10:57:47.540825 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.540715    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bacefdfe-6069-449d-b814-4f7a24915796" containerName="extract"
Sep 17 10:57:47.544680 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.544664    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9"
Sep 17 10:57:47.551313 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.551293    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 10:57:47.551313 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.551301    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 10:57:47.551830 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.551817    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7z6jb\""
Sep 17 10:57:47.562464 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.562441    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9"]
Sep 17 10:57:47.717235 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.717201    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gj2s\" (UniqueName: \"kubernetes.io/projected/fcebd772-fb28-44b2-97dc-f50ef3655497-kube-api-access-2gj2s\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9\" (UID: \"fcebd772-fb28-44b2-97dc-f50ef3655497\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9"
Sep 17 10:57:47.717364 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.717239    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fcebd772-fb28-44b2-97dc-f50ef3655497-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9\" (UID: \"fcebd772-fb28-44b2-97dc-f50ef3655497\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9"
Sep 17 10:57:47.717364 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.717273    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fcebd772-fb28-44b2-97dc-f50ef3655497-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9\" (UID: \"fcebd772-fb28-44b2-97dc-f50ef3655497\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9"
Sep 17 10:57:47.817933 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.817828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fcebd772-fb28-44b2-97dc-f50ef3655497-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9\" (UID: \"fcebd772-fb28-44b2-97dc-f50ef3655497\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9"
Sep 17 10:57:47.817933 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.817871    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fcebd772-fb28-44b2-97dc-f50ef3655497-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9\" (UID: \"fcebd772-fb28-44b2-97dc-f50ef3655497\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9"
Sep 17 10:57:47.818150 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.817981    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gj2s\" (UniqueName: \"kubernetes.io/projected/fcebd772-fb28-44b2-97dc-f50ef3655497-kube-api-access-2gj2s\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9\" (UID: \"fcebd772-fb28-44b2-97dc-f50ef3655497\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9"
Sep 17 10:57:47.818235 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.818213    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fcebd772-fb28-44b2-97dc-f50ef3655497-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9\" (UID: \"fcebd772-fb28-44b2-97dc-f50ef3655497\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9"
Sep 17 10:57:47.818294 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.818281    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fcebd772-fb28-44b2-97dc-f50ef3655497-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9\" (UID: \"fcebd772-fb28-44b2-97dc-f50ef3655497\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9"
Sep 17 10:57:47.842653 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.842624    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gj2s\" (UniqueName: \"kubernetes.io/projected/fcebd772-fb28-44b2-97dc-f50ef3655497-kube-api-access-2gj2s\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9\" (UID: \"fcebd772-fb28-44b2-97dc-f50ef3655497\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9"
Sep 17 10:57:47.853311 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:47.853291    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9"
Sep 17 10:57:48.136497 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:48.136473    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9"]
Sep 17 10:57:48.138364 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:57:48.138328    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcebd772_fb28_44b2_97dc_f50ef3655497.slice/crio-3e985c5f29049a7c919e4d51e03f29e9fc0a1618ad2418cb41e001a19a2faf20 WatchSource:0}: Error finding container 3e985c5f29049a7c919e4d51e03f29e9fc0a1618ad2418cb41e001a19a2faf20: Status 404 returned error can't find the container with id 3e985c5f29049a7c919e4d51e03f29e9fc0a1618ad2418cb41e001a19a2faf20
Sep 17 10:57:48.272217 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:48.272186    2564 generic.go:356] "Generic (PLEG): container finished" podID="fcebd772-fb28-44b2-97dc-f50ef3655497" containerID="1c0a16a76d08336418bd88d59d940c48ad9aa44a922105074dda01e944844dbf" exitCode=0
Sep 17 10:57:48.272333 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:48.272271    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9" event={"ID":"fcebd772-fb28-44b2-97dc-f50ef3655497","Type":"ContainerDied","Data":"1c0a16a76d08336418bd88d59d940c48ad9aa44a922105074dda01e944844dbf"}
Sep 17 10:57:48.272333 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:48.272322    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9" event={"ID":"fcebd772-fb28-44b2-97dc-f50ef3655497","Type":"ContainerStarted","Data":"3e985c5f29049a7c919e4d51e03f29e9fc0a1618ad2418cb41e001a19a2faf20"}
Sep 17 10:57:49.211052 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.210980    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"]
Sep 17 10:57:49.214213 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.214196    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.218254 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.218229    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 17 10:57:49.218387 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.218235    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 17 10:57:49.218387 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.218366    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ff97h\""
Sep 17 10:57:49.222706 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.222687    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 17 10:57:49.222937 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.222924    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 17 10:57:49.255503 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.255482    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"]
Sep 17 10:57:49.278053 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.278028    2564 generic.go:356] "Generic (PLEG): container finished" podID="fcebd772-fb28-44b2-97dc-f50ef3655497" containerID="9064abddb1af8b29c980b95630912c38652a615cd12e647114b8b9b262911e91" exitCode=0
Sep 17 10:57:49.278157 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.278097    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9" event={"ID":"fcebd772-fb28-44b2-97dc-f50ef3655497","Type":"ContainerDied","Data":"9064abddb1af8b29c980b95630912c38652a615cd12e647114b8b9b262911e91"}
Sep 17 10:57:49.328748 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.328725    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8109edee-2793-485d-8998-9b4dc165396f-manifests\") pod \"opendatahub-operator-controller-manager-c54c4bd97-bw97f\" (UID: \"8109edee-2793-485d-8998-9b4dc165396f\") " pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.328860 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.328770    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8109edee-2793-485d-8998-9b4dc165396f-webhook-cert\") pod \"opendatahub-operator-controller-manager-c54c4bd97-bw97f\" (UID: \"8109edee-2793-485d-8998-9b4dc165396f\") " pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.328860 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.328802    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8109edee-2793-485d-8998-9b4dc165396f-tmp\") pod \"opendatahub-operator-controller-manager-c54c4bd97-bw97f\" (UID: \"8109edee-2793-485d-8998-9b4dc165396f\") " pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.328860 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.328855    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8109edee-2793-485d-8998-9b4dc165396f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c54c4bd97-bw97f\" (UID: \"8109edee-2793-485d-8998-9b4dc165396f\") " pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.329017 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.328911    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg6gw\" (UniqueName: \"kubernetes.io/projected/8109edee-2793-485d-8998-9b4dc165396f-kube-api-access-zg6gw\") pod \"opendatahub-operator-controller-manager-c54c4bd97-bw97f\" (UID: \"8109edee-2793-485d-8998-9b4dc165396f\") " pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.429603 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.429563    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8109edee-2793-485d-8998-9b4dc165396f-manifests\") pod \"opendatahub-operator-controller-manager-c54c4bd97-bw97f\" (UID: \"8109edee-2793-485d-8998-9b4dc165396f\") " pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.429791 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.429614    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8109edee-2793-485d-8998-9b4dc165396f-webhook-cert\") pod \"opendatahub-operator-controller-manager-c54c4bd97-bw97f\" (UID: \"8109edee-2793-485d-8998-9b4dc165396f\") " pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.429791 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.429633    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8109edee-2793-485d-8998-9b4dc165396f-tmp\") pod \"opendatahub-operator-controller-manager-c54c4bd97-bw97f\" (UID: \"8109edee-2793-485d-8998-9b4dc165396f\") " pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.429791 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.429665    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8109edee-2793-485d-8998-9b4dc165396f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c54c4bd97-bw97f\" (UID: \"8109edee-2793-485d-8998-9b4dc165396f\") " pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.429791 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.429690    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg6gw\" (UniqueName: \"kubernetes.io/projected/8109edee-2793-485d-8998-9b4dc165396f-kube-api-access-zg6gw\") pod \"opendatahub-operator-controller-manager-c54c4bd97-bw97f\" (UID: \"8109edee-2793-485d-8998-9b4dc165396f\") " pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.430124 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.430057    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8109edee-2793-485d-8998-9b4dc165396f-manifests\") pod \"opendatahub-operator-controller-manager-c54c4bd97-bw97f\" (UID: \"8109edee-2793-485d-8998-9b4dc165396f\") " pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.432363 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.432328    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8109edee-2793-485d-8998-9b4dc165396f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c54c4bd97-bw97f\" (UID: \"8109edee-2793-485d-8998-9b4dc165396f\") " pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.432363 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.432354    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8109edee-2793-485d-8998-9b4dc165396f-tmp\") pod \"opendatahub-operator-controller-manager-c54c4bd97-bw97f\" (UID: \"8109edee-2793-485d-8998-9b4dc165396f\") " pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.432563 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.432521    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8109edee-2793-485d-8998-9b4dc165396f-webhook-cert\") pod \"opendatahub-operator-controller-manager-c54c4bd97-bw97f\" (UID: \"8109edee-2793-485d-8998-9b4dc165396f\") " pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.439772 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.439739    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg6gw\" (UniqueName: \"kubernetes.io/projected/8109edee-2793-485d-8998-9b4dc165396f-kube-api-access-zg6gw\") pod \"opendatahub-operator-controller-manager-c54c4bd97-bw97f\" (UID: \"8109edee-2793-485d-8998-9b4dc165396f\") " pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.525264 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.525238    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:49.824303 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:49.824129    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"]
Sep 17 10:57:49.827026 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:57:49.827003    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8109edee_2793_485d_8998_9b4dc165396f.slice/crio-ef281d86d110697cc16bcfaa4edd574df0b2567c18c1c67ab4edc80973630602 WatchSource:0}: Error finding container ef281d86d110697cc16bcfaa4edd574df0b2567c18c1c67ab4edc80973630602: Status 404 returned error can't find the container with id ef281d86d110697cc16bcfaa4edd574df0b2567c18c1c67ab4edc80973630602
Sep 17 10:57:50.284154 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:50.284121    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f" event={"ID":"8109edee-2793-485d-8998-9b4dc165396f","Type":"ContainerStarted","Data":"ef281d86d110697cc16bcfaa4edd574df0b2567c18c1c67ab4edc80973630602"}
Sep 17 10:57:50.286425 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:50.286401    2564 generic.go:356] "Generic (PLEG): container finished" podID="fcebd772-fb28-44b2-97dc-f50ef3655497" containerID="41bc1398c7999fe7bb9f457720cb91480a18ef4e907c158b0d6388a630467d2e" exitCode=0
Sep 17 10:57:50.286549 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:50.286464    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9" event={"ID":"fcebd772-fb28-44b2-97dc-f50ef3655497","Type":"ContainerDied","Data":"41bc1398c7999fe7bb9f457720cb91480a18ef4e907c158b0d6388a630467d2e"}
Sep 17 10:57:52.942370 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:52.942346    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9"
Sep 17 10:57:53.059029 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:53.058997    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gj2s\" (UniqueName: \"kubernetes.io/projected/fcebd772-fb28-44b2-97dc-f50ef3655497-kube-api-access-2gj2s\") pod \"fcebd772-fb28-44b2-97dc-f50ef3655497\" (UID: \"fcebd772-fb28-44b2-97dc-f50ef3655497\") "
Sep 17 10:57:53.059179 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:53.059062    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fcebd772-fb28-44b2-97dc-f50ef3655497-bundle\") pod \"fcebd772-fb28-44b2-97dc-f50ef3655497\" (UID: \"fcebd772-fb28-44b2-97dc-f50ef3655497\") "
Sep 17 10:57:53.059179 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:53.059092    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fcebd772-fb28-44b2-97dc-f50ef3655497-util\") pod \"fcebd772-fb28-44b2-97dc-f50ef3655497\" (UID: \"fcebd772-fb28-44b2-97dc-f50ef3655497\") "
Sep 17 10:57:53.059943 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:53.059882    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcebd772-fb28-44b2-97dc-f50ef3655497-bundle" (OuterVolumeSpecName: "bundle") pod "fcebd772-fb28-44b2-97dc-f50ef3655497" (UID: "fcebd772-fb28-44b2-97dc-f50ef3655497"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:57:53.061301 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:53.061275    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcebd772-fb28-44b2-97dc-f50ef3655497-kube-api-access-2gj2s" (OuterVolumeSpecName: "kube-api-access-2gj2s") pod "fcebd772-fb28-44b2-97dc-f50ef3655497" (UID: "fcebd772-fb28-44b2-97dc-f50ef3655497"). InnerVolumeSpecName "kube-api-access-2gj2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 10:57:53.067772 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:53.067744    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcebd772-fb28-44b2-97dc-f50ef3655497-util" (OuterVolumeSpecName: "util") pod "fcebd772-fb28-44b2-97dc-f50ef3655497" (UID: "fcebd772-fb28-44b2-97dc-f50ef3655497"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:57:53.159604 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:53.159560    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fcebd772-fb28-44b2-97dc-f50ef3655497-bundle\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:57:53.159604 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:53.159580    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fcebd772-fb28-44b2-97dc-f50ef3655497-util\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:57:53.159604 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:53.159589    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gj2s\" (UniqueName: \"kubernetes.io/projected/fcebd772-fb28-44b2-97dc-f50ef3655497-kube-api-access-2gj2s\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:57:53.303376 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:53.303353    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9"
Sep 17 10:57:53.303472 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:53.303352    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h4rp9" event={"ID":"fcebd772-fb28-44b2-97dc-f50ef3655497","Type":"ContainerDied","Data":"3e985c5f29049a7c919e4d51e03f29e9fc0a1618ad2418cb41e001a19a2faf20"}
Sep 17 10:57:53.303534 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:53.303481    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e985c5f29049a7c919e4d51e03f29e9fc0a1618ad2418cb41e001a19a2faf20"
Sep 17 10:57:54.312968 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:54.312935    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f" event={"ID":"8109edee-2793-485d-8998-9b4dc165396f","Type":"ContainerStarted","Data":"5a4c7db5912d963fdf5cc7738265c9a55eacb294f92ba3c388e24070eb16b0fb"}
Sep 17 10:57:55.319013 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:55.318979    2564 generic.go:356] "Generic (PLEG): container finished" podID="8109edee-2793-485d-8998-9b4dc165396f" containerID="5a4c7db5912d963fdf5cc7738265c9a55eacb294f92ba3c388e24070eb16b0fb" exitCode=0
Sep 17 10:57:55.319493 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:55.319027    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f" event={"ID":"8109edee-2793-485d-8998-9b4dc165396f","Type":"ContainerDied","Data":"5a4c7db5912d963fdf5cc7738265c9a55eacb294f92ba3c388e24070eb16b0fb"}
Sep 17 10:57:56.325544 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:56.325514    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f" event={"ID":"8109edee-2793-485d-8998-9b4dc165396f","Type":"ContainerStarted","Data":"40265a4868635d4464258a6c8b5d24f96551bd688db5a93ab86fafe1afd49d91"}
Sep 17 10:57:56.325866 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:56.325616    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:57:56.348133 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:57:56.348083    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f" podStartSLOduration=2.4010507739999998 podStartE2EDuration="7.348068682s" totalImagesPullingTime="3.951404585s" totalInitContainerRuntime="995.613323ms" isStatefulPod=false podCreationTimestamp="2026-09-17 10:57:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:57:56.346718304 +0000 UTC m=+271.663081459" watchObservedRunningTime="2026-09-17 10:57:56.348068682 +0000 UTC m=+271.664431837"
Sep 17 10:58:06.156374 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.156340    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"]
Sep 17 10:58:06.156775 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.156655    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcebd772-fb28-44b2-97dc-f50ef3655497" containerName="util"
Sep 17 10:58:06.156775 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.156667    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcebd772-fb28-44b2-97dc-f50ef3655497" containerName="util"
Sep 17 10:58:06.156775 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.156676    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcebd772-fb28-44b2-97dc-f50ef3655497" containerName="pull"
Sep 17 10:58:06.156775 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.156682    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcebd772-fb28-44b2-97dc-f50ef3655497" containerName="pull"
Sep 17 10:58:06.156775 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.156700    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcebd772-fb28-44b2-97dc-f50ef3655497" containerName="extract"
Sep 17 10:58:06.156775 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.156705    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcebd772-fb28-44b2-97dc-f50ef3655497" containerName="extract"
Sep 17 10:58:06.156775 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.156751    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcebd772-fb28-44b2-97dc-f50ef3655497" containerName="extract"
Sep 17 10:58:06.164660 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.164645    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"
Sep 17 10:58:06.174602 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.174583    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 17 10:58:06.174999 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.174985    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 17 10:58:06.175098 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.175082    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-djmd9\""
Sep 17 10:58:06.175098 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.175092    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 17 10:58:06.175176 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.175112    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 17 10:58:06.175217 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.175190    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 17 10:58:06.191764 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.191737    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"]
Sep 17 10:58:06.253776 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.253755    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4ac624b0-4480-4a91-a803-a1dd59b1ada2-manager-config\") pod \"lws-controller-manager-7bb4f45b46-sg2qj\" (UID: \"4ac624b0-4480-4a91-a803-a1dd59b1ada2\") " pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"
Sep 17 10:58:06.253925 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.253799    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ac624b0-4480-4a91-a803-a1dd59b1ada2-metrics-cert\") pod \"lws-controller-manager-7bb4f45b46-sg2qj\" (UID: \"4ac624b0-4480-4a91-a803-a1dd59b1ada2\") " pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"
Sep 17 10:58:06.253925 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.253823    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ac624b0-4480-4a91-a803-a1dd59b1ada2-cert\") pod \"lws-controller-manager-7bb4f45b46-sg2qj\" (UID: \"4ac624b0-4480-4a91-a803-a1dd59b1ada2\") " pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"
Sep 17 10:58:06.253925 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.253839    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g766n\" (UniqueName: \"kubernetes.io/projected/4ac624b0-4480-4a91-a803-a1dd59b1ada2-kube-api-access-g766n\") pod \"lws-controller-manager-7bb4f45b46-sg2qj\" (UID: \"4ac624b0-4480-4a91-a803-a1dd59b1ada2\") " pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"
Sep 17 10:58:06.354501 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.354475    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ac624b0-4480-4a91-a803-a1dd59b1ada2-metrics-cert\") pod \"lws-controller-manager-7bb4f45b46-sg2qj\" (UID: \"4ac624b0-4480-4a91-a803-a1dd59b1ada2\") " pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"
Sep 17 10:58:06.354602 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.354508    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ac624b0-4480-4a91-a803-a1dd59b1ada2-cert\") pod \"lws-controller-manager-7bb4f45b46-sg2qj\" (UID: \"4ac624b0-4480-4a91-a803-a1dd59b1ada2\") " pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"
Sep 17 10:58:06.354602 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.354527    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g766n\" (UniqueName: \"kubernetes.io/projected/4ac624b0-4480-4a91-a803-a1dd59b1ada2-kube-api-access-g766n\") pod \"lws-controller-manager-7bb4f45b46-sg2qj\" (UID: \"4ac624b0-4480-4a91-a803-a1dd59b1ada2\") " pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"
Sep 17 10:58:06.354602 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.354573    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4ac624b0-4480-4a91-a803-a1dd59b1ada2-manager-config\") pod \"lws-controller-manager-7bb4f45b46-sg2qj\" (UID: \"4ac624b0-4480-4a91-a803-a1dd59b1ada2\") " pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"
Sep 17 10:58:06.355241 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.355224    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4ac624b0-4480-4a91-a803-a1dd59b1ada2-manager-config\") pod \"lws-controller-manager-7bb4f45b46-sg2qj\" (UID: \"4ac624b0-4480-4a91-a803-a1dd59b1ada2\") " pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"
Sep 17 10:58:06.356966 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.356946    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ac624b0-4480-4a91-a803-a1dd59b1ada2-metrics-cert\") pod \"lws-controller-manager-7bb4f45b46-sg2qj\" (UID: \"4ac624b0-4480-4a91-a803-a1dd59b1ada2\") " pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"
Sep 17 10:58:06.357075 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.357058    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ac624b0-4480-4a91-a803-a1dd59b1ada2-cert\") pod \"lws-controller-manager-7bb4f45b46-sg2qj\" (UID: \"4ac624b0-4480-4a91-a803-a1dd59b1ada2\") " pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"
Sep 17 10:58:06.362581 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.362562    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g766n\" (UniqueName: \"kubernetes.io/projected/4ac624b0-4480-4a91-a803-a1dd59b1ada2-kube-api-access-g766n\") pod \"lws-controller-manager-7bb4f45b46-sg2qj\" (UID: \"4ac624b0-4480-4a91-a803-a1dd59b1ada2\") " pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"
Sep 17 10:58:06.473335 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.473273    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"
Sep 17 10:58:06.747563 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:06.747481    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"]
Sep 17 10:58:06.753437 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:58:06.753408    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ac624b0_4480_4a91_a803_a1dd59b1ada2.slice/crio-5364ea295e704cf0aec03eaf817c5adc0387040af648edbca3834ab2560e0a77 WatchSource:0}: Error finding container 5364ea295e704cf0aec03eaf817c5adc0387040af648edbca3834ab2560e0a77: Status 404 returned error can't find the container with id 5364ea295e704cf0aec03eaf817c5adc0387040af648edbca3834ab2560e0a77
Sep 17 10:58:07.332073 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:07.332044    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-c54c4bd97-bw97f"
Sep 17 10:58:07.378568 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:07.378529    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj" event={"ID":"4ac624b0-4480-4a91-a803-a1dd59b1ada2","Type":"ContainerStarted","Data":"5364ea295e704cf0aec03eaf817c5adc0387040af648edbca3834ab2560e0a77"}
Sep 17 10:58:08.384273 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:08.384239    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj" event={"ID":"4ac624b0-4480-4a91-a803-a1dd59b1ada2","Type":"ContainerStarted","Data":"ed906c3bccaa733200842f9869fbd4ca8487dda00372453cbdb7bf04accfcb13"}
Sep 17 10:58:08.384652 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:08.384347    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"
Sep 17 10:58:08.406459 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:08.406411    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj" podStartSLOduration=1.277240468 podStartE2EDuration="2.406397813s" totalImagesPullingTime="1.129157345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:58:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:58:08.404329205 +0000 UTC m=+283.720692371" watchObservedRunningTime="2026-09-17 10:58:08.406397813 +0000 UTC m=+283.722760967"
Sep 17 10:58:19.367335 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.367302    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"]
Sep 17 10:58:19.371042 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.371026    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.383940 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:58:19.383915    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-129-205.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-129-205.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Sep 17 10:58:19.384039 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:58:19.384019    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-gw-ca-root-cert\" is forbidden: User \"system:node:ip-10-0-129-205.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-129-205.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" type="*v1.ConfigMap"
Sep 17 10:58:19.384209 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.384197    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-qsjgt\""
Sep 17 10:58:19.384403 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.384389    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 17 10:58:19.389379 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.389360    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7bb4f45b46-sg2qj"
Sep 17 10:58:19.445567 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.445541    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e730ef5-89da-4980-a9a1-77060acf61eb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.445667 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.445578    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e730ef5-89da-4980-a9a1-77060acf61eb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.445667 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.445596    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwhxf\" (UniqueName: \"kubernetes.io/projected/0e730ef5-89da-4980-a9a1-77060acf61eb-kube-api-access-kwhxf\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.445753 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.445741    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e730ef5-89da-4980-a9a1-77060acf61eb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.445794 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.445760    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e730ef5-89da-4980-a9a1-77060acf61eb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.445794 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.445784    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e730ef5-89da-4980-a9a1-77060acf61eb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.445864 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.445844    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e730ef5-89da-4980-a9a1-77060acf61eb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.445983 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.445943    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e730ef5-89da-4980-a9a1-77060acf61eb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.445983 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.445965    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e730ef5-89da-4980-a9a1-77060acf61eb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.456020 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.455985    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"]
Sep 17 10:58:19.547335 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.547304    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e730ef5-89da-4980-a9a1-77060acf61eb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.547335 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.547338    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwhxf\" (UniqueName: \"kubernetes.io/projected/0e730ef5-89da-4980-a9a1-77060acf61eb-kube-api-access-kwhxf\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.547519 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.547362    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e730ef5-89da-4980-a9a1-77060acf61eb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.547519 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.547379    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e730ef5-89da-4980-a9a1-77060acf61eb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.547519 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.547400    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e730ef5-89da-4980-a9a1-77060acf61eb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.547519 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.547427    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e730ef5-89da-4980-a9a1-77060acf61eb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.547519 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.547460    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e730ef5-89da-4980-a9a1-77060acf61eb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.547519 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.547482    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e730ef5-89da-4980-a9a1-77060acf61eb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.547730 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.547555    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e730ef5-89da-4980-a9a1-77060acf61eb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.547782 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.547763    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e730ef5-89da-4980-a9a1-77060acf61eb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.547832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.547812    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e730ef5-89da-4980-a9a1-77060acf61eb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.547932 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.547886    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e730ef5-89da-4980-a9a1-77060acf61eb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.547979 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.547955    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e730ef5-89da-4980-a9a1-77060acf61eb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.549989 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.549955    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e730ef5-89da-4980-a9a1-77060acf61eb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.550116 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.550083    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e730ef5-89da-4980-a9a1-77060acf61eb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:19.559151 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:19.559091    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e730ef5-89da-4980-a9a1-77060acf61eb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:20.441485 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:20.441449    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mnwq4"]
Sep 17 10:58:20.444984 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:20.444969    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mnwq4"
Sep 17 10:58:20.447742 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:20.447726    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 17 10:58:20.448435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:20.448422    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-8zm6h\""
Sep 17 10:58:20.460738 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:20.460708    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mnwq4"]
Sep 17 10:58:20.548512 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:58:20.548486    2564 configmap.go:193] Couldn't get configMap openshift-ingress/openshift-gw-ca-root-cert: failed to sync configmap cache: timed out waiting for the condition
Sep 17 10:58:20.548630 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:58:20.548553    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e730ef5-89da-4980-a9a1-77060acf61eb-istiod-ca-cert podName:0e730ef5-89da-4980-a9a1-77060acf61eb nodeName:}" failed. No retries permitted until 2026-09-17 10:58:21.048535101 +0000 UTC m=+296.364898235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "istiod-ca-cert" (UniqueName: "kubernetes.io/configmap/0e730ef5-89da-4980-a9a1-77060acf61eb-istiod-ca-cert") pod "data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd" (UID: "0e730ef5-89da-4980-a9a1-77060acf61eb") : failed to sync configmap cache: timed out waiting for the condition
Sep 17 10:58:20.556226 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:20.556202    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swvnw\" (UniqueName: \"kubernetes.io/projected/d1c78d62-da7a-4cd9-98bd-377a059e6613-kube-api-access-swvnw\") pod \"odh-model-controller-7998c67dc4-mnwq4\" (UID: \"d1c78d62-da7a-4cd9-98bd-377a059e6613\") " pod="opendatahub/odh-model-controller-7998c67dc4-mnwq4"
Sep 17 10:58:20.556300 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:20.556244    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1c78d62-da7a-4cd9-98bd-377a059e6613-cert\") pod \"odh-model-controller-7998c67dc4-mnwq4\" (UID: \"d1c78d62-da7a-4cd9-98bd-377a059e6613\") " pod="opendatahub/odh-model-controller-7998c67dc4-mnwq4"
Sep 17 10:58:20.559147 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:58:20.559132    2564 projected.go:291] Couldn't get configMap openshift-ingress/openshift-service-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Sep 17 10:58:20.559198 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:58:20.559150    2564 projected.go:196] Error preparing data for projected volume kube-api-access-kwhxf for pod openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd: failed to sync configmap cache: timed out waiting for the condition
Sep 17 10:58:20.559232 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:58:20.559202    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0e730ef5-89da-4980-a9a1-77060acf61eb-kube-api-access-kwhxf podName:0e730ef5-89da-4980-a9a1-77060acf61eb nodeName:}" failed. No retries permitted until 2026-09-17 10:58:21.059188287 +0000 UTC m=+296.375551420 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kwhxf" (UniqueName: "kubernetes.io/projected/0e730ef5-89da-4980-a9a1-77060acf61eb-kube-api-access-kwhxf") pod "data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd" (UID: "0e730ef5-89da-4980-a9a1-77060acf61eb") : failed to sync configmap cache: timed out waiting for the condition
Sep 17 10:58:20.657356 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:20.657327    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swvnw\" (UniqueName: \"kubernetes.io/projected/d1c78d62-da7a-4cd9-98bd-377a059e6613-kube-api-access-swvnw\") pod \"odh-model-controller-7998c67dc4-mnwq4\" (UID: \"d1c78d62-da7a-4cd9-98bd-377a059e6613\") " pod="opendatahub/odh-model-controller-7998c67dc4-mnwq4"
Sep 17 10:58:20.657476 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:20.657365    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1c78d62-da7a-4cd9-98bd-377a059e6613-cert\") pod \"odh-model-controller-7998c67dc4-mnwq4\" (UID: \"d1c78d62-da7a-4cd9-98bd-377a059e6613\") " pod="opendatahub/odh-model-controller-7998c67dc4-mnwq4"
Sep 17 10:58:20.657512 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:58:20.657476    2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 17 10:58:20.657561 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:58:20.657541    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1c78d62-da7a-4cd9-98bd-377a059e6613-cert podName:d1c78d62-da7a-4cd9-98bd-377a059e6613 nodeName:}" failed. No retries permitted until 2026-09-17 10:58:21.157522554 +0000 UTC m=+296.473885701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d1c78d62-da7a-4cd9-98bd-377a059e6613-cert") pod "odh-model-controller-7998c67dc4-mnwq4" (UID: "d1c78d62-da7a-4cd9-98bd-377a059e6613") : secret "odh-model-controller-webhook-cert" not found
Sep 17 10:58:20.671339 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:20.671318    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swvnw\" (UniqueName: \"kubernetes.io/projected/d1c78d62-da7a-4cd9-98bd-377a059e6613-kube-api-access-swvnw\") pod \"odh-model-controller-7998c67dc4-mnwq4\" (UID: \"d1c78d62-da7a-4cd9-98bd-377a059e6613\") " pod="opendatahub/odh-model-controller-7998c67dc4-mnwq4"
Sep 17 10:58:20.916878 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:20.916849    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 17 10:58:20.965907 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:20.965884    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 17 10:58:21.060495 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.060463    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e730ef5-89da-4980-a9a1-77060acf61eb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:21.060658 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.060504    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwhxf\" (UniqueName: \"kubernetes.io/projected/0e730ef5-89da-4980-a9a1-77060acf61eb-kube-api-access-kwhxf\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:21.061175 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.061157    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e730ef5-89da-4980-a9a1-77060acf61eb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:21.063248 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.063231    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwhxf\" (UniqueName: \"kubernetes.io/projected/0e730ef5-89da-4980-a9a1-77060acf61eb-kube-api-access-kwhxf\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd\" (UID: \"0e730ef5-89da-4980-a9a1-77060acf61eb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:21.161221 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.161194    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1c78d62-da7a-4cd9-98bd-377a059e6613-cert\") pod \"odh-model-controller-7998c67dc4-mnwq4\" (UID: \"d1c78d62-da7a-4cd9-98bd-377a059e6613\") " pod="opendatahub/odh-model-controller-7998c67dc4-mnwq4"
Sep 17 10:58:21.163621 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.163603    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1c78d62-da7a-4cd9-98bd-377a059e6613-cert\") pod \"odh-model-controller-7998c67dc4-mnwq4\" (UID: \"d1c78d62-da7a-4cd9-98bd-377a059e6613\") " pod="opendatahub/odh-model-controller-7998c67dc4-mnwq4"
Sep 17 10:58:21.180137 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.180079    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:21.224774 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.224745    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7kqvf"]
Sep 17 10:58:21.228585 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.228556    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7kqvf"
Sep 17 10:58:21.234319 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.233499    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-54f55\""
Sep 17 10:58:21.234319 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.234006    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 17 10:58:21.251313 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.244082    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7kqvf"]
Sep 17 10:58:21.262174 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.261981    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b5cd969-4bc5-4f8e-89e3-1e36d3598b36-tls-certs\") pod \"model-serving-api-54d89bd5fc-7kqvf\" (UID: \"2b5cd969-4bc5-4f8e-89e3-1e36d3598b36\") " pod="opendatahub/model-serving-api-54d89bd5fc-7kqvf"
Sep 17 10:58:21.262174 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.262106    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vtwf\" (UniqueName: \"kubernetes.io/projected/2b5cd969-4bc5-4f8e-89e3-1e36d3598b36-kube-api-access-8vtwf\") pod \"model-serving-api-54d89bd5fc-7kqvf\" (UID: \"2b5cd969-4bc5-4f8e-89e3-1e36d3598b36\") " pod="opendatahub/model-serving-api-54d89bd5fc-7kqvf"
Sep 17 10:58:21.354741 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.354700    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mnwq4"
Sep 17 10:58:21.362711 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.362684    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b5cd969-4bc5-4f8e-89e3-1e36d3598b36-tls-certs\") pod \"model-serving-api-54d89bd5fc-7kqvf\" (UID: \"2b5cd969-4bc5-4f8e-89e3-1e36d3598b36\") " pod="opendatahub/model-serving-api-54d89bd5fc-7kqvf"
Sep 17 10:58:21.362821 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.362765    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vtwf\" (UniqueName: \"kubernetes.io/projected/2b5cd969-4bc5-4f8e-89e3-1e36d3598b36-kube-api-access-8vtwf\") pod \"model-serving-api-54d89bd5fc-7kqvf\" (UID: \"2b5cd969-4bc5-4f8e-89e3-1e36d3598b36\") " pod="opendatahub/model-serving-api-54d89bd5fc-7kqvf"
Sep 17 10:58:21.362821 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:58:21.362803    2564 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 17 10:58:21.362922 ip-10-0-129-205 kubenswrapper[2564]: E0917 10:58:21.362880    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b5cd969-4bc5-4f8e-89e3-1e36d3598b36-tls-certs podName:2b5cd969-4bc5-4f8e-89e3-1e36d3598b36 nodeName:}" failed. No retries permitted until 2026-09-17 10:58:21.862858158 +0000 UTC m=+297.179221294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2b5cd969-4bc5-4f8e-89e3-1e36d3598b36-tls-certs") pod "model-serving-api-54d89bd5fc-7kqvf" (UID: "2b5cd969-4bc5-4f8e-89e3-1e36d3598b36") : secret "model-serving-api-tls" not found
Sep 17 10:58:21.372584 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.372552    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vtwf\" (UniqueName: \"kubernetes.io/projected/2b5cd969-4bc5-4f8e-89e3-1e36d3598b36-kube-api-access-8vtwf\") pod \"model-serving-api-54d89bd5fc-7kqvf\" (UID: \"2b5cd969-4bc5-4f8e-89e3-1e36d3598b36\") " pod="opendatahub/model-serving-api-54d89bd5fc-7kqvf"
Sep 17 10:58:21.492311 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.492233    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"]
Sep 17 10:58:21.494593 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:58:21.494569    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e730ef5_89da_4980_a9a1_77060acf61eb.slice/crio-a77e72108d393c0ca29a9c0b202f0688df37423ca77fcb3ac90a6799ebc9ec48 WatchSource:0}: Error finding container a77e72108d393c0ca29a9c0b202f0688df37423ca77fcb3ac90a6799ebc9ec48: Status 404 returned error can't find the container with id a77e72108d393c0ca29a9c0b202f0688df37423ca77fcb3ac90a6799ebc9ec48
Sep 17 10:58:21.636656 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.636619    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mnwq4"]
Sep 17 10:58:21.642928 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:58:21.642883    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c78d62_da7a_4cd9_98bd_377a059e6613.slice/crio-d951fa0bd339dfb7028f2ab439b1f366e09d7342f30fed0fac2bd59dada49b03 WatchSource:0}: Error finding container d951fa0bd339dfb7028f2ab439b1f366e09d7342f30fed0fac2bd59dada49b03: Status 404 returned error can't find the container with id d951fa0bd339dfb7028f2ab439b1f366e09d7342f30fed0fac2bd59dada49b03
Sep 17 10:58:21.867928 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.867833    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b5cd969-4bc5-4f8e-89e3-1e36d3598b36-tls-certs\") pod \"model-serving-api-54d89bd5fc-7kqvf\" (UID: \"2b5cd969-4bc5-4f8e-89e3-1e36d3598b36\") " pod="opendatahub/model-serving-api-54d89bd5fc-7kqvf"
Sep 17 10:58:21.870356 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:21.870334    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b5cd969-4bc5-4f8e-89e3-1e36d3598b36-tls-certs\") pod \"model-serving-api-54d89bd5fc-7kqvf\" (UID: \"2b5cd969-4bc5-4f8e-89e3-1e36d3598b36\") " pod="opendatahub/model-serving-api-54d89bd5fc-7kqvf"
Sep 17 10:58:22.155030 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:22.154958    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7kqvf"
Sep 17 10:58:22.434120 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:22.433989    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7kqvf"]
Sep 17 10:58:22.439939 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:58:22.439516    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b5cd969_4bc5_4f8e_89e3_1e36d3598b36.slice/crio-5bcc5b82990b32484767afaf7801f7fb6697214520ce372397e41b9bd0f58f83 WatchSource:0}: Error finding container 5bcc5b82990b32484767afaf7801f7fb6697214520ce372397e41b9bd0f58f83: Status 404 returned error can't find the container with id 5bcc5b82990b32484767afaf7801f7fb6697214520ce372397e41b9bd0f58f83
Sep 17 10:58:22.450358 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:22.450328    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mnwq4" event={"ID":"d1c78d62-da7a-4cd9-98bd-377a059e6613","Type":"ContainerStarted","Data":"d951fa0bd339dfb7028f2ab439b1f366e09d7342f30fed0fac2bd59dada49b03"}
Sep 17 10:58:22.452456 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:22.452429    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd" event={"ID":"0e730ef5-89da-4980-a9a1-77060acf61eb","Type":"ContainerStarted","Data":"a77e72108d393c0ca29a9c0b202f0688df37423ca77fcb3ac90a6799ebc9ec48"}
Sep 17 10:58:23.461006 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:23.460881    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7kqvf" event={"ID":"2b5cd969-4bc5-4f8e-89e3-1e36d3598b36","Type":"ContainerStarted","Data":"5bcc5b82990b32484767afaf7801f7fb6697214520ce372397e41b9bd0f58f83"}
Sep 17 10:58:26.695067 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:26.695019    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 10:58:26.695542 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:26.695123    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 10:58:26.695542 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:26.695173    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 10:58:26.727456 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:26.727431    2564 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 10:58:27.072541 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.072506    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"]
Sep 17 10:58:27.074918 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.074881    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.077669 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.077648    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 17 10:58:27.077785 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.077648    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 17 10:58:27.077785 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.077648    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 17 10:58:27.078131 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.078116    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4wmdh\""
Sep 17 10:58:27.090865 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.090834    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"]
Sep 17 10:58:27.111169 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.111126    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/156dfbf0-52a9-4789-baa4-9bf47c07c620-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.111169 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.111169    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/156dfbf0-52a9-4789-baa4-9bf47c07c620-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.111402 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.111201    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/156dfbf0-52a9-4789-baa4-9bf47c07c620-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.111402 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.111232    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh496\" (UniqueName: \"kubernetes.io/projected/156dfbf0-52a9-4789-baa4-9bf47c07c620-kube-api-access-dh496\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.111402 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.111320    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/156dfbf0-52a9-4789-baa4-9bf47c07c620-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.111402 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.111359    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/156dfbf0-52a9-4789-baa4-9bf47c07c620-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.111568 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.111405    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/156dfbf0-52a9-4789-baa4-9bf47c07c620-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.212709 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.212668    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/156dfbf0-52a9-4789-baa4-9bf47c07c620-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.212941 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.212751    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/156dfbf0-52a9-4789-baa4-9bf47c07c620-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.212941 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.212790    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/156dfbf0-52a9-4789-baa4-9bf47c07c620-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.212941 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.212878    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/156dfbf0-52a9-4789-baa4-9bf47c07c620-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.212941 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.212929    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/156dfbf0-52a9-4789-baa4-9bf47c07c620-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.213168 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.212956    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/156dfbf0-52a9-4789-baa4-9bf47c07c620-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.213168 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.212993    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh496\" (UniqueName: \"kubernetes.io/projected/156dfbf0-52a9-4789-baa4-9bf47c07c620-kube-api-access-dh496\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.214077 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.213984    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/156dfbf0-52a9-4789-baa4-9bf47c07c620-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.215507 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.215474    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/156dfbf0-52a9-4789-baa4-9bf47c07c620-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.215639 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.215574    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/156dfbf0-52a9-4789-baa4-9bf47c07c620-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.215639 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.215625    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/156dfbf0-52a9-4789-baa4-9bf47c07c620-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.215752 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.215669    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/156dfbf0-52a9-4789-baa4-9bf47c07c620-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.225349 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.225320    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh496\" (UniqueName: \"kubernetes.io/projected/156dfbf0-52a9-4789-baa4-9bf47c07c620-kube-api-access-dh496\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.226243 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.226216    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/156dfbf0-52a9-4789-baa4-9bf47c07c620-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-dqjkh\" (UID: \"156dfbf0-52a9-4789-baa4-9bf47c07c620\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.384509 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.384412    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:27.482415 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.482384    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7kqvf" event={"ID":"2b5cd969-4bc5-4f8e-89e3-1e36d3598b36","Type":"ContainerStarted","Data":"e3e7f96208d146ef4a0bc59c49229e1e554af43a4b1d1fd5b246403a2c5ed9e6"}
Sep 17 10:58:27.482571 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.482449    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-7kqvf"
Sep 17 10:58:27.484190 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.484166    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mnwq4" event={"ID":"d1c78d62-da7a-4cd9-98bd-377a059e6613","Type":"ContainerStarted","Data":"96a267c8b580a691aef6d7d4929fa660e89106ffcccd7b37782318a82717eb50"}
Sep 17 10:58:27.484308 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.484228    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-mnwq4"
Sep 17 10:58:27.485822 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.485800    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd" event={"ID":"0e730ef5-89da-4980-a9a1-77060acf61eb","Type":"ContainerStarted","Data":"f4169b8fb78311eaae5a0e0470f6789841ba02ea453b162c2c784427168a3f2b"}
Sep 17 10:58:27.505536 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.505495    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-7kqvf" podStartSLOduration=2.245550813 podStartE2EDuration="6.505481306s" totalImagesPullingTime="4.259930493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:58:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:58:27.504408119 +0000 UTC m=+302.820771275" watchObservedRunningTime="2026-09-17 10:58:27.505481306 +0000 UTC m=+302.821844460"
Sep 17 10:58:27.540229 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.540174    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd" podStartSLOduration=3.341857567 podStartE2EDuration="8.540153598s" totalImagesPullingTime="5.198296031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:58:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:58:27.537601419 +0000 UTC m=+302.853964602" watchObservedRunningTime="2026-09-17 10:58:27.540153598 +0000 UTC m=+302.856516754"
Sep 17 10:58:27.583465 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.583375    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-mnwq4" podStartSLOduration=2.524092966 podStartE2EDuration="7.583357256s" totalImagesPullingTime="5.05926429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:58:27.581192189 +0000 UTC m=+302.897555346" watchObservedRunningTime="2026-09-17 10:58:27.583357256 +0000 UTC m=+302.899720413"
Sep 17 10:58:27.692512 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.692481    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"]
Sep 17 10:58:27.693861 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:58:27.693837    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod156dfbf0_52a9_4789_baa4_9bf47c07c620.slice/crio-90d6b8e4e40421f9940f7963940c6a112af62ed58518f38d54df5bd136502e05 WatchSource:0}: Error finding container 90d6b8e4e40421f9940f7963940c6a112af62ed58518f38d54df5bd136502e05: Status 404 returned error can't find the container with id 90d6b8e4e40421f9940f7963940c6a112af62ed58518f38d54df5bd136502e05
Sep 17 10:58:27.695997 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:27.695982    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 10:58:28.180373 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:28.180345    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:28.184833 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:28.184809    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:28.491947 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:28.491884    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh" event={"ID":"156dfbf0-52a9-4789-baa4-9bf47c07c620","Type":"ContainerStarted","Data":"90d6b8e4e40421f9940f7963940c6a112af62ed58518f38d54df5bd136502e05"}
Sep 17 10:58:28.492200 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:28.492182    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:28.493134 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:28.493113    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd"
Sep 17 10:58:30.534225 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:30.534175    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 10:58:30.534615 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:30.534260    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 10:58:31.509169 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:31.509133    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh" event={"ID":"156dfbf0-52a9-4789-baa4-9bf47c07c620","Type":"ContainerStarted","Data":"dfe8d949e7b444e9d282943af4ed4d3b7781b748d0b0369bb264f6f050dd2366"}
Sep 17 10:58:31.509364 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:31.509342    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:31.511036 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:31.511017    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh"
Sep 17 10:58:31.576339 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:31.576294    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-dqjkh" podStartSLOduration=1.738502105 podStartE2EDuration="4.576280612s" totalImagesPullingTime="2.837778507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 10:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 10:58:31.558766421 +0000 UTC m=+306.875129575" watchObservedRunningTime="2026-09-17 10:58:31.576280612 +0000 UTC m=+306.892643768"
Sep 17 10:58:38.494371 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:38.494342    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-mnwq4"
Sep 17 10:58:38.496498 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:58:38.496476    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-7kqvf"
Sep 17 10:59:27.194557 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.194526    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz"]
Sep 17 10:59:27.197016 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.196999    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz"
Sep 17 10:59:27.199456 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.199430    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 10:59:27.199593 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.199463    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7z6jb\""
Sep 17 10:59:27.199593 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.199432    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 10:59:27.209758 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.209734    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz"]
Sep 17 10:59:27.294381 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.294356    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr"]
Sep 17 10:59:27.296597 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.296581    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr"
Sep 17 10:59:27.304665 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.304642    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eba0e36c-051e-4b97-b453-9305f67ff6e6-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz\" (UID: \"eba0e36c-051e-4b97-b453-9305f67ff6e6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz"
Sep 17 10:59:27.304859 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.304839    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b44jb\" (UniqueName: \"kubernetes.io/projected/eba0e36c-051e-4b97-b453-9305f67ff6e6-kube-api-access-b44jb\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz\" (UID: \"eba0e36c-051e-4b97-b453-9305f67ff6e6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz"
Sep 17 10:59:27.304991 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.304892    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eba0e36c-051e-4b97-b453-9305f67ff6e6-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz\" (UID: \"eba0e36c-051e-4b97-b453-9305f67ff6e6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz"
Sep 17 10:59:27.306447 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.306427    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr"]
Sep 17 10:59:27.391056 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.391010    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn"]
Sep 17 10:59:27.393401 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.393385    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn"
Sep 17 10:59:27.403637 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.403612    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn"]
Sep 17 10:59:27.405891 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.405864    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9713c405-ff92-43ba-a406-778a848cd2e8-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr\" (UID: \"9713c405-ff92-43ba-a406-778a848cd2e8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr"
Sep 17 10:59:27.406018 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.405945    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ljhf\" (UniqueName: \"kubernetes.io/projected/9713c405-ff92-43ba-a406-778a848cd2e8-kube-api-access-9ljhf\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr\" (UID: \"9713c405-ff92-43ba-a406-778a848cd2e8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr"
Sep 17 10:59:27.406018 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.405993    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eba0e36c-051e-4b97-b453-9305f67ff6e6-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz\" (UID: \"eba0e36c-051e-4b97-b453-9305f67ff6e6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz"
Sep 17 10:59:27.406095 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.406037    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b44jb\" (UniqueName: \"kubernetes.io/projected/eba0e36c-051e-4b97-b453-9305f67ff6e6-kube-api-access-b44jb\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz\" (UID: \"eba0e36c-051e-4b97-b453-9305f67ff6e6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz"
Sep 17 10:59:27.406095 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.406071    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9713c405-ff92-43ba-a406-778a848cd2e8-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr\" (UID: \"9713c405-ff92-43ba-a406-778a848cd2e8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr"
Sep 17 10:59:27.406180 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.406101    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eba0e36c-051e-4b97-b453-9305f67ff6e6-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz\" (UID: \"eba0e36c-051e-4b97-b453-9305f67ff6e6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz"
Sep 17 10:59:27.406435 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.406414    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eba0e36c-051e-4b97-b453-9305f67ff6e6-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz\" (UID: \"eba0e36c-051e-4b97-b453-9305f67ff6e6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz"
Sep 17 10:59:27.406502 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.406432    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eba0e36c-051e-4b97-b453-9305f67ff6e6-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz\" (UID: \"eba0e36c-051e-4b97-b453-9305f67ff6e6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz"
Sep 17 10:59:27.414715 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.414694    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b44jb\" (UniqueName: \"kubernetes.io/projected/eba0e36c-051e-4b97-b453-9305f67ff6e6-kube-api-access-b44jb\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz\" (UID: \"eba0e36c-051e-4b97-b453-9305f67ff6e6\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz"
Sep 17 10:59:27.491295 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.491269    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd"]
Sep 17 10:59:27.493651 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.493636    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd"
Sep 17 10:59:27.504512 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.504488    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd"]
Sep 17 10:59:27.505798 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.505778    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz"
Sep 17 10:59:27.507090 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.507072    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn\" (UID: \"7f3afda8-4b97-4821-a1df-a2d8b585a2f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn"
Sep 17 10:59:27.507157 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.507099    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fxk7\" (UniqueName: \"kubernetes.io/projected/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-kube-api-access-4fxk7\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn\" (UID: \"7f3afda8-4b97-4821-a1df-a2d8b585a2f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn"
Sep 17 10:59:27.507157 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.507143    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9713c405-ff92-43ba-a406-778a848cd2e8-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr\" (UID: \"9713c405-ff92-43ba-a406-778a848cd2e8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr"
Sep 17 10:59:27.507231 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.507183    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn\" (UID: \"7f3afda8-4b97-4821-a1df-a2d8b585a2f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn"
Sep 17 10:59:27.507231 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.507218    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9713c405-ff92-43ba-a406-778a848cd2e8-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr\" (UID: \"9713c405-ff92-43ba-a406-778a848cd2e8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr"
Sep 17 10:59:27.507308 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.507247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ljhf\" (UniqueName: \"kubernetes.io/projected/9713c405-ff92-43ba-a406-778a848cd2e8-kube-api-access-9ljhf\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr\" (UID: \"9713c405-ff92-43ba-a406-778a848cd2e8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr"
Sep 17 10:59:27.507848 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.507569    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9713c405-ff92-43ba-a406-778a848cd2e8-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr\" (UID: \"9713c405-ff92-43ba-a406-778a848cd2e8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr"
Sep 17 10:59:27.507848 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.507577    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9713c405-ff92-43ba-a406-778a848cd2e8-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr\" (UID: \"9713c405-ff92-43ba-a406-778a848cd2e8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr"
Sep 17 10:59:27.517782 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.517761    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ljhf\" (UniqueName: \"kubernetes.io/projected/9713c405-ff92-43ba-a406-778a848cd2e8-kube-api-access-9ljhf\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr\" (UID: \"9713c405-ff92-43ba-a406-778a848cd2e8\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr"
Sep 17 10:59:27.605839 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.605797    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr"
Sep 17 10:59:27.608371 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.608156    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8d58430-ee5e-4f4a-9790-8cad5cde3596-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd\" (UID: \"a8d58430-ee5e-4f4a-9790-8cad5cde3596\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd"
Sep 17 10:59:27.608371 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.608202    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-777h7\" (UniqueName: \"kubernetes.io/projected/a8d58430-ee5e-4f4a-9790-8cad5cde3596-kube-api-access-777h7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd\" (UID: \"a8d58430-ee5e-4f4a-9790-8cad5cde3596\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd"
Sep 17 10:59:27.608371 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.608275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn\" (UID: \"7f3afda8-4b97-4821-a1df-a2d8b585a2f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn"
Sep 17 10:59:27.608371 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.608313    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8d58430-ee5e-4f4a-9790-8cad5cde3596-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd\" (UID: \"a8d58430-ee5e-4f4a-9790-8cad5cde3596\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd"
Sep 17 10:59:27.608371 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.608365    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn\" (UID: \"7f3afda8-4b97-4821-a1df-a2d8b585a2f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn"
Sep 17 10:59:27.608653 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.608384    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fxk7\" (UniqueName: \"kubernetes.io/projected/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-kube-api-access-4fxk7\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn\" (UID: \"7f3afda8-4b97-4821-a1df-a2d8b585a2f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn"
Sep 17 10:59:27.608699 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.608652    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn\" (UID: \"7f3afda8-4b97-4821-a1df-a2d8b585a2f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn"
Sep 17 10:59:27.608738 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.608686    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn\" (UID: \"7f3afda8-4b97-4821-a1df-a2d8b585a2f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn"
Sep 17 10:59:27.617344 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.617324    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fxk7\" (UniqueName: \"kubernetes.io/projected/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-kube-api-access-4fxk7\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn\" (UID: \"7f3afda8-4b97-4821-a1df-a2d8b585a2f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn"
Sep 17 10:59:27.702357 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.702330    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn"
Sep 17 10:59:27.709320 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.709292    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8d58430-ee5e-4f4a-9790-8cad5cde3596-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd\" (UID: \"a8d58430-ee5e-4f4a-9790-8cad5cde3596\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd"
Sep 17 10:59:27.709426 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.709349    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8d58430-ee5e-4f4a-9790-8cad5cde3596-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd\" (UID: \"a8d58430-ee5e-4f4a-9790-8cad5cde3596\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd"
Sep 17 10:59:27.709426 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.709376    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-777h7\" (UniqueName: \"kubernetes.io/projected/a8d58430-ee5e-4f4a-9790-8cad5cde3596-kube-api-access-777h7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd\" (UID: \"a8d58430-ee5e-4f4a-9790-8cad5cde3596\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd"
Sep 17 10:59:27.709641 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.709622    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8d58430-ee5e-4f4a-9790-8cad5cde3596-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd\" (UID: \"a8d58430-ee5e-4f4a-9790-8cad5cde3596\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd"
Sep 17 10:59:27.709711 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.709640    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8d58430-ee5e-4f4a-9790-8cad5cde3596-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd\" (UID: \"a8d58430-ee5e-4f4a-9790-8cad5cde3596\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd"
Sep 17 10:59:27.719723 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.719700    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-777h7\" (UniqueName: \"kubernetes.io/projected/a8d58430-ee5e-4f4a-9790-8cad5cde3596-kube-api-access-777h7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd\" (UID: \"a8d58430-ee5e-4f4a-9790-8cad5cde3596\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd"
Sep 17 10:59:27.785438 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.785408    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz"]
Sep 17 10:59:27.787138 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:59:27.787108    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeba0e36c_051e_4b97_b453_9305f67ff6e6.slice/crio-ab1ff611c76ae69a09d3c974224060d8cc73d4f009ee432c487230c98cf5d7b2 WatchSource:0}: Error finding container ab1ff611c76ae69a09d3c974224060d8cc73d4f009ee432c487230c98cf5d7b2: Status 404 returned error can't find the container with id ab1ff611c76ae69a09d3c974224060d8cc73d4f009ee432c487230c98cf5d7b2
Sep 17 10:59:27.803886 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.803866    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd"
Sep 17 10:59:27.885109 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.884950    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr"]
Sep 17 10:59:27.887454 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:59:27.887423    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9713c405_ff92_43ba_a406_778a848cd2e8.slice/crio-8b5c3f1f430dd492ee11fd7aaa5db6d825c7d2b6bea479ce7a37e5d64eb343ac WatchSource:0}: Error finding container 8b5c3f1f430dd492ee11fd7aaa5db6d825c7d2b6bea479ce7a37e5d64eb343ac: Status 404 returned error can't find the container with id 8b5c3f1f430dd492ee11fd7aaa5db6d825c7d2b6bea479ce7a37e5d64eb343ac
Sep 17 10:59:27.980297 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:27.980267    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn"]
Sep 17 10:59:27.984648 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:59:27.984615    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f3afda8_4b97_4821_a1df_a2d8b585a2f2.slice/crio-4cf8d06b668098f683592862a2e86241641a6e1476e13ed603940000bf5b101e WatchSource:0}: Error finding container 4cf8d06b668098f683592862a2e86241641a6e1476e13ed603940000bf5b101e: Status 404 returned error can't find the container with id 4cf8d06b668098f683592862a2e86241641a6e1476e13ed603940000bf5b101e
Sep 17 10:59:28.079583 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:28.079562    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd"]
Sep 17 10:59:28.081317 ip-10-0-129-205 kubenswrapper[2564]: W0917 10:59:28.081291    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8d58430_ee5e_4f4a_9790_8cad5cde3596.slice/crio-b0728ccac7996deb55c32ff1d6e6016c68fa720546b78d5f0ea6c8c9d564e94f WatchSource:0}: Error finding container b0728ccac7996deb55c32ff1d6e6016c68fa720546b78d5f0ea6c8c9d564e94f: Status 404 returned error can't find the container with id b0728ccac7996deb55c32ff1d6e6016c68fa720546b78d5f0ea6c8c9d564e94f
Sep 17 10:59:28.783950 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:28.783890    2564 generic.go:356] "Generic (PLEG): container finished" podID="a8d58430-ee5e-4f4a-9790-8cad5cde3596" containerID="20f4867f89f3a4fd57590afef0816be8a920f6cdce08674f90e12272bca4120f" exitCode=0
Sep 17 10:59:28.784378 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:28.783997    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd" event={"ID":"a8d58430-ee5e-4f4a-9790-8cad5cde3596","Type":"ContainerDied","Data":"20f4867f89f3a4fd57590afef0816be8a920f6cdce08674f90e12272bca4120f"}
Sep 17 10:59:28.784378 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:28.784055    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd" event={"ID":"a8d58430-ee5e-4f4a-9790-8cad5cde3596","Type":"ContainerStarted","Data":"b0728ccac7996deb55c32ff1d6e6016c68fa720546b78d5f0ea6c8c9d564e94f"}
Sep 17 10:59:28.785739 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:28.785714    2564 generic.go:356] "Generic (PLEG): container finished" podID="9713c405-ff92-43ba-a406-778a848cd2e8" containerID="778a0934dd80dae0ba32bb0a4a2fe874a1a57b025b9761afc18a800900a17d19" exitCode=0
Sep 17 10:59:28.785831 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:28.785798    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr" event={"ID":"9713c405-ff92-43ba-a406-778a848cd2e8","Type":"ContainerDied","Data":"778a0934dd80dae0ba32bb0a4a2fe874a1a57b025b9761afc18a800900a17d19"}
Sep 17 10:59:28.785929 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:28.785846    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr" event={"ID":"9713c405-ff92-43ba-a406-778a848cd2e8","Type":"ContainerStarted","Data":"8b5c3f1f430dd492ee11fd7aaa5db6d825c7d2b6bea479ce7a37e5d64eb343ac"}
Sep 17 10:59:28.787605 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:28.787586    2564 generic.go:356] "Generic (PLEG): container finished" podID="7f3afda8-4b97-4821-a1df-a2d8b585a2f2" containerID="e4d10228bfa2c727534bb3dd50e539421934ad912195fcbbb97a8a5bce94cd31" exitCode=0
Sep 17 10:59:28.787697 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:28.787648    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn" event={"ID":"7f3afda8-4b97-4821-a1df-a2d8b585a2f2","Type":"ContainerDied","Data":"e4d10228bfa2c727534bb3dd50e539421934ad912195fcbbb97a8a5bce94cd31"}
Sep 17 10:59:28.787697 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:28.787692    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn" event={"ID":"7f3afda8-4b97-4821-a1df-a2d8b585a2f2","Type":"ContainerStarted","Data":"4cf8d06b668098f683592862a2e86241641a6e1476e13ed603940000bf5b101e"}
Sep 17 10:59:28.789563 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:28.789543    2564 generic.go:356] "Generic (PLEG): container finished" podID="eba0e36c-051e-4b97-b453-9305f67ff6e6" containerID="49a6bad4d18b2bae647df5985ce1b9f966c5448660adb1bb5601ef5f0a75027b" exitCode=0
Sep 17 10:59:28.789646 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:28.789583    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz" event={"ID":"eba0e36c-051e-4b97-b453-9305f67ff6e6","Type":"ContainerDied","Data":"49a6bad4d18b2bae647df5985ce1b9f966c5448660adb1bb5601ef5f0a75027b"}
Sep 17 10:59:28.789646 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:28.789615    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz" event={"ID":"eba0e36c-051e-4b97-b453-9305f67ff6e6","Type":"ContainerStarted","Data":"ab1ff611c76ae69a09d3c974224060d8cc73d4f009ee432c487230c98cf5d7b2"}
Sep 17 10:59:29.795889 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:29.795820    2564 generic.go:356] "Generic (PLEG): container finished" podID="7f3afda8-4b97-4821-a1df-a2d8b585a2f2" containerID="ab9a2da197d1b81d2a622525e366a96f4e05f564b4747a55d74acdd0de3ce4ee" exitCode=0
Sep 17 10:59:29.796273 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:29.795930    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn" event={"ID":"7f3afda8-4b97-4821-a1df-a2d8b585a2f2","Type":"ContainerDied","Data":"ab9a2da197d1b81d2a622525e366a96f4e05f564b4747a55d74acdd0de3ce4ee"}
Sep 17 10:59:29.797959 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:29.797932    2564 generic.go:356] "Generic (PLEG): container finished" podID="eba0e36c-051e-4b97-b453-9305f67ff6e6" containerID="aa550ec8c66f73f387e756a955f2a3b9d5ea4a003a3d7797f9af2893aa494381" exitCode=0
Sep 17 10:59:29.798058 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:29.798022    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz" event={"ID":"eba0e36c-051e-4b97-b453-9305f67ff6e6","Type":"ContainerDied","Data":"aa550ec8c66f73f387e756a955f2a3b9d5ea4a003a3d7797f9af2893aa494381"}
Sep 17 10:59:30.804502 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:30.804469    2564 generic.go:356] "Generic (PLEG): container finished" podID="9713c405-ff92-43ba-a406-778a848cd2e8" containerID="b988350f941288d666dd95627a73edb98af2e6a401b0264457e916d743095b2b" exitCode=0
Sep 17 10:59:30.804979 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:30.804562    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr" event={"ID":"9713c405-ff92-43ba-a406-778a848cd2e8","Type":"ContainerDied","Data":"b988350f941288d666dd95627a73edb98af2e6a401b0264457e916d743095b2b"}
Sep 17 10:59:30.807075 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:30.807053    2564 generic.go:356] "Generic (PLEG): container finished" podID="7f3afda8-4b97-4821-a1df-a2d8b585a2f2" containerID="34d28d12951dd74a6ba0b315fb95cda97448755c60a80f09bc122fe0d015e5b0" exitCode=0
Sep 17 10:59:30.807152 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:30.807090    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn" event={"ID":"7f3afda8-4b97-4821-a1df-a2d8b585a2f2","Type":"ContainerDied","Data":"34d28d12951dd74a6ba0b315fb95cda97448755c60a80f09bc122fe0d015e5b0"}
Sep 17 10:59:30.809453 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:30.809431    2564 generic.go:356] "Generic (PLEG): container finished" podID="eba0e36c-051e-4b97-b453-9305f67ff6e6" containerID="309e23bd27cb2f0c29ded509048d6008c151f3b1dd2af9103bef8b4839ffa898" exitCode=0
Sep 17 10:59:30.809531 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:30.809471    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz" event={"ID":"eba0e36c-051e-4b97-b453-9305f67ff6e6","Type":"ContainerDied","Data":"309e23bd27cb2f0c29ded509048d6008c151f3b1dd2af9103bef8b4839ffa898"}
Sep 17 10:59:31.816422 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:31.816388    2564 generic.go:356] "Generic (PLEG): container finished" podID="9713c405-ff92-43ba-a406-778a848cd2e8" containerID="a6524bd287ea846b04828ce10d2001e2a3a08ed08eaa971c8d9ec4ca08d1c8a1" exitCode=0
Sep 17 10:59:31.816832 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:31.816474    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr" event={"ID":"9713c405-ff92-43ba-a406-778a848cd2e8","Type":"ContainerDied","Data":"a6524bd287ea846b04828ce10d2001e2a3a08ed08eaa971c8d9ec4ca08d1c8a1"}
Sep 17 10:59:31.947522 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:31.947494    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn"
Sep 17 10:59:31.981273 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:31.981252    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz"
Sep 17 10:59:32.045245 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.045218    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fxk7\" (UniqueName: \"kubernetes.io/projected/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-kube-api-access-4fxk7\") pod \"7f3afda8-4b97-4821-a1df-a2d8b585a2f2\" (UID: \"7f3afda8-4b97-4821-a1df-a2d8b585a2f2\") "
Sep 17 10:59:32.045358 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.045267    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eba0e36c-051e-4b97-b453-9305f67ff6e6-util\") pod \"eba0e36c-051e-4b97-b453-9305f67ff6e6\" (UID: \"eba0e36c-051e-4b97-b453-9305f67ff6e6\") "
Sep 17 10:59:32.045358 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.045327    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eba0e36c-051e-4b97-b453-9305f67ff6e6-bundle\") pod \"eba0e36c-051e-4b97-b453-9305f67ff6e6\" (UID: \"eba0e36c-051e-4b97-b453-9305f67ff6e6\") "
Sep 17 10:59:32.045358 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.045342    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-util\") pod \"7f3afda8-4b97-4821-a1df-a2d8b585a2f2\" (UID: \"7f3afda8-4b97-4821-a1df-a2d8b585a2f2\") "
Sep 17 10:59:32.045467 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.045368    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b44jb\" (UniqueName: \"kubernetes.io/projected/eba0e36c-051e-4b97-b453-9305f67ff6e6-kube-api-access-b44jb\") pod \"eba0e36c-051e-4b97-b453-9305f67ff6e6\" (UID: \"eba0e36c-051e-4b97-b453-9305f67ff6e6\") "
Sep 17 10:59:32.045467 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.045414    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-bundle\") pod \"7f3afda8-4b97-4821-a1df-a2d8b585a2f2\" (UID: \"7f3afda8-4b97-4821-a1df-a2d8b585a2f2\") "
Sep 17 10:59:32.045999 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.045957    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba0e36c-051e-4b97-b453-9305f67ff6e6-bundle" (OuterVolumeSpecName: "bundle") pod "eba0e36c-051e-4b97-b453-9305f67ff6e6" (UID: "eba0e36c-051e-4b97-b453-9305f67ff6e6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:59:32.046124 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.046024    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-bundle" (OuterVolumeSpecName: "bundle") pod "7f3afda8-4b97-4821-a1df-a2d8b585a2f2" (UID: "7f3afda8-4b97-4821-a1df-a2d8b585a2f2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:59:32.047702 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.047666    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eba0e36c-051e-4b97-b453-9305f67ff6e6-kube-api-access-b44jb" (OuterVolumeSpecName: "kube-api-access-b44jb") pod "eba0e36c-051e-4b97-b453-9305f67ff6e6" (UID: "eba0e36c-051e-4b97-b453-9305f67ff6e6"). InnerVolumeSpecName "kube-api-access-b44jb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 10:59:32.047803 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.047785    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-kube-api-access-4fxk7" (OuterVolumeSpecName: "kube-api-access-4fxk7") pod "7f3afda8-4b97-4821-a1df-a2d8b585a2f2" (UID: "7f3afda8-4b97-4821-a1df-a2d8b585a2f2"). InnerVolumeSpecName "kube-api-access-4fxk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 10:59:32.051357 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.051324    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba0e36c-051e-4b97-b453-9305f67ff6e6-util" (OuterVolumeSpecName: "util") pod "eba0e36c-051e-4b97-b453-9305f67ff6e6" (UID: "eba0e36c-051e-4b97-b453-9305f67ff6e6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:59:32.051886 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.051859    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-util" (OuterVolumeSpecName: "util") pod "7f3afda8-4b97-4821-a1df-a2d8b585a2f2" (UID: "7f3afda8-4b97-4821-a1df-a2d8b585a2f2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:59:32.145986 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.145934    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eba0e36c-051e-4b97-b453-9305f67ff6e6-bundle\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:59:32.145986 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.145954    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-util\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:59:32.145986 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.145963    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b44jb\" (UniqueName: \"kubernetes.io/projected/eba0e36c-051e-4b97-b453-9305f67ff6e6-kube-api-access-b44jb\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:59:32.145986 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.145972    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-bundle\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:59:32.145986 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.145980    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fxk7\" (UniqueName: \"kubernetes.io/projected/7f3afda8-4b97-4821-a1df-a2d8b585a2f2-kube-api-access-4fxk7\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:59:32.145986 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.145987    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eba0e36c-051e-4b97-b453-9305f67ff6e6-util\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:59:32.823632 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.823554    2564 generic.go:356] "Generic (PLEG): container finished" podID="a8d58430-ee5e-4f4a-9790-8cad5cde3596" containerID="09482dc41ea7815c086c4b217be093c229a37499b76a789a41d9cd6bcadc8fd1" exitCode=0
Sep 17 10:59:32.824030 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.823645    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd" event={"ID":"a8d58430-ee5e-4f4a-9790-8cad5cde3596","Type":"ContainerDied","Data":"09482dc41ea7815c086c4b217be093c229a37499b76a789a41d9cd6bcadc8fd1"}
Sep 17 10:59:32.825819 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.825790    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn" event={"ID":"7f3afda8-4b97-4821-a1df-a2d8b585a2f2","Type":"ContainerDied","Data":"4cf8d06b668098f683592862a2e86241641a6e1476e13ed603940000bf5b101e"}
Sep 17 10:59:32.825933 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.825826    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4cf8d06b668098f683592862a2e86241641a6e1476e13ed603940000bf5b101e"
Sep 17 10:59:32.825933 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.825800    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7s65xn"
Sep 17 10:59:32.828116 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.828099    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz"
Sep 17 10:59:32.828205 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.828122    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ck68wz" event={"ID":"eba0e36c-051e-4b97-b453-9305f67ff6e6","Type":"ContainerDied","Data":"ab1ff611c76ae69a09d3c974224060d8cc73d4f009ee432c487230c98cf5d7b2"}
Sep 17 10:59:32.828205 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.828168    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab1ff611c76ae69a09d3c974224060d8cc73d4f009ee432c487230c98cf5d7b2"
Sep 17 10:59:32.966491 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:32.966474    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr"
Sep 17 10:59:33.055141 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:33.055115    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9713c405-ff92-43ba-a406-778a848cd2e8-util\") pod \"9713c405-ff92-43ba-a406-778a848cd2e8\" (UID: \"9713c405-ff92-43ba-a406-778a848cd2e8\") "
Sep 17 10:59:33.055295 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:33.055166    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ljhf\" (UniqueName: \"kubernetes.io/projected/9713c405-ff92-43ba-a406-778a848cd2e8-kube-api-access-9ljhf\") pod \"9713c405-ff92-43ba-a406-778a848cd2e8\" (UID: \"9713c405-ff92-43ba-a406-778a848cd2e8\") "
Sep 17 10:59:33.055295 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:33.055232    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9713c405-ff92-43ba-a406-778a848cd2e8-bundle\") pod \"9713c405-ff92-43ba-a406-778a848cd2e8\" (UID: \"9713c405-ff92-43ba-a406-778a848cd2e8\") "
Sep 17 10:59:33.055945 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:33.055892    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9713c405-ff92-43ba-a406-778a848cd2e8-bundle" (OuterVolumeSpecName: "bundle") pod "9713c405-ff92-43ba-a406-778a848cd2e8" (UID: "9713c405-ff92-43ba-a406-778a848cd2e8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:59:33.057433 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:33.057410    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9713c405-ff92-43ba-a406-778a848cd2e8-kube-api-access-9ljhf" (OuterVolumeSpecName: "kube-api-access-9ljhf") pod "9713c405-ff92-43ba-a406-778a848cd2e8" (UID: "9713c405-ff92-43ba-a406-778a848cd2e8"). InnerVolumeSpecName "kube-api-access-9ljhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 10:59:33.060568 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:33.060545    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9713c405-ff92-43ba-a406-778a848cd2e8-util" (OuterVolumeSpecName: "util") pod "9713c405-ff92-43ba-a406-778a848cd2e8" (UID: "9713c405-ff92-43ba-a406-778a848cd2e8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:59:33.156449 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:33.156423    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9713c405-ff92-43ba-a406-778a848cd2e8-bundle\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:59:33.156449 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:33.156445    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9713c405-ff92-43ba-a406-778a848cd2e8-util\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:59:33.156576 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:33.156455    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ljhf\" (UniqueName: \"kubernetes.io/projected/9713c405-ff92-43ba-a406-778a848cd2e8-kube-api-access-9ljhf\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:59:33.835484 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:33.835453    2564 generic.go:356] "Generic (PLEG): container finished" podID="a8d58430-ee5e-4f4a-9790-8cad5cde3596" containerID="a3f0ff88c6676e37d75487f80df2bddb4d82ee9a264f37a817713374a84f83be" exitCode=0
Sep 17 10:59:33.835943 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:33.835533    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd" event={"ID":"a8d58430-ee5e-4f4a-9790-8cad5cde3596","Type":"ContainerDied","Data":"a3f0ff88c6676e37d75487f80df2bddb4d82ee9a264f37a817713374a84f83be"}
Sep 17 10:59:33.837380 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:33.837364    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr"
Sep 17 10:59:33.837541 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:33.837361    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adkrqr" event={"ID":"9713c405-ff92-43ba-a406-778a848cd2e8","Type":"ContainerDied","Data":"8b5c3f1f430dd492ee11fd7aaa5db6d825c7d2b6bea479ce7a37e5d64eb343ac"}
Sep 17 10:59:33.837541 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:33.837475    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b5c3f1f430dd492ee11fd7aaa5db6d825c7d2b6bea479ce7a37e5d64eb343ac"
Sep 17 10:59:34.975534 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:34.975511    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd"
Sep 17 10:59:35.071304 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:35.071275    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8d58430-ee5e-4f4a-9790-8cad5cde3596-util\") pod \"a8d58430-ee5e-4f4a-9790-8cad5cde3596\" (UID: \"a8d58430-ee5e-4f4a-9790-8cad5cde3596\") "
Sep 17 10:59:35.071447 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:35.071377    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8d58430-ee5e-4f4a-9790-8cad5cde3596-bundle\") pod \"a8d58430-ee5e-4f4a-9790-8cad5cde3596\" (UID: \"a8d58430-ee5e-4f4a-9790-8cad5cde3596\") "
Sep 17 10:59:35.071447 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:35.071422    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-777h7\" (UniqueName: \"kubernetes.io/projected/a8d58430-ee5e-4f4a-9790-8cad5cde3596-kube-api-access-777h7\") pod \"a8d58430-ee5e-4f4a-9790-8cad5cde3596\" (UID: \"a8d58430-ee5e-4f4a-9790-8cad5cde3596\") "
Sep 17 10:59:35.071801 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:35.071776    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8d58430-ee5e-4f4a-9790-8cad5cde3596-bundle" (OuterVolumeSpecName: "bundle") pod "a8d58430-ee5e-4f4a-9790-8cad5cde3596" (UID: "a8d58430-ee5e-4f4a-9790-8cad5cde3596"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:59:35.073629 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:35.073602    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8d58430-ee5e-4f4a-9790-8cad5cde3596-kube-api-access-777h7" (OuterVolumeSpecName: "kube-api-access-777h7") pod "a8d58430-ee5e-4f4a-9790-8cad5cde3596" (UID: "a8d58430-ee5e-4f4a-9790-8cad5cde3596"). InnerVolumeSpecName "kube-api-access-777h7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 10:59:35.075962 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:35.075944    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8d58430-ee5e-4f4a-9790-8cad5cde3596-util" (OuterVolumeSpecName: "util") pod "a8d58430-ee5e-4f4a-9790-8cad5cde3596" (UID: "a8d58430-ee5e-4f4a-9790-8cad5cde3596"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 10:59:35.172131 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:35.172111    2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8d58430-ee5e-4f4a-9790-8cad5cde3596-bundle\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:59:35.172131 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:35.172132    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-777h7\" (UniqueName: \"kubernetes.io/projected/a8d58430-ee5e-4f4a-9790-8cad5cde3596-kube-api-access-777h7\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:59:35.172270 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:35.172141    2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8d58430-ee5e-4f4a-9790-8cad5cde3596-util\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 10:59:35.853303 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:35.853269    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd" event={"ID":"a8d58430-ee5e-4f4a-9790-8cad5cde3596","Type":"ContainerDied","Data":"b0728ccac7996deb55c32ff1d6e6016c68fa720546b78d5f0ea6c8c9d564e94f"}
Sep 17 10:59:35.853470 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:35.853322    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0728ccac7996deb55c32ff1d6e6016c68fa720546b78d5f0ea6c8c9d564e94f"
Sep 17 10:59:35.853470 ip-10-0-129-205 kubenswrapper[2564]: I0917 10:59:35.853284    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6hj4xd"
Sep 17 11:00:05.599437 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:05.599403    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d4c6d598d-wh4dm"]
Sep 17 11:00:30.618400 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.618348    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d4c6d598d-wh4dm" podUID="1c0bf612-05ae-400f-9511-af776c601549" containerName="console" containerID="cri-o://cd4156e9ba5b4bedfdfcb9d46bbbc818ffe21d93911bd62e56442c5198346ac6" gracePeriod=15
Sep 17 11:00:30.859298 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.859271    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 11:00:30.906097 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.906041    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-oauth-serving-cert\") pod \"1c0bf612-05ae-400f-9511-af776c601549\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") "
Sep 17 11:00:30.906097 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.906074    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-trusted-ca-bundle\") pod \"1c0bf612-05ae-400f-9511-af776c601549\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") "
Sep 17 11:00:30.906285 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.906119    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dw7zr\" (UniqueName: \"kubernetes.io/projected/1c0bf612-05ae-400f-9511-af776c601549-kube-api-access-dw7zr\") pod \"1c0bf612-05ae-400f-9511-af776c601549\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") "
Sep 17 11:00:30.906285 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.906143    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-service-ca\") pod \"1c0bf612-05ae-400f-9511-af776c601549\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") "
Sep 17 11:00:30.906285 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.906166    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-console-config\") pod \"1c0bf612-05ae-400f-9511-af776c601549\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") "
Sep 17 11:00:30.906285 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.906205    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c0bf612-05ae-400f-9511-af776c601549-console-serving-cert\") pod \"1c0bf612-05ae-400f-9511-af776c601549\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") "
Sep 17 11:00:30.906285 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.906254    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c0bf612-05ae-400f-9511-af776c601549-console-oauth-config\") pod \"1c0bf612-05ae-400f-9511-af776c601549\" (UID: \"1c0bf612-05ae-400f-9511-af776c601549\") "
Sep 17 11:00:30.906542 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.906451    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1c0bf612-05ae-400f-9511-af776c601549" (UID: "1c0bf612-05ae-400f-9511-af776c601549"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:00:30.906542 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.906515    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1c0bf612-05ae-400f-9511-af776c601549" (UID: "1c0bf612-05ae-400f-9511-af776c601549"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:00:30.906633 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.906561    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-oauth-serving-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:00:30.906850 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.906823    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-console-config" (OuterVolumeSpecName: "console-config") pod "1c0bf612-05ae-400f-9511-af776c601549" (UID: "1c0bf612-05ae-400f-9511-af776c601549"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:00:30.907037 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.907018    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-service-ca" (OuterVolumeSpecName: "service-ca") pod "1c0bf612-05ae-400f-9511-af776c601549" (UID: "1c0bf612-05ae-400f-9511-af776c601549"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:00:30.908755 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.908735    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c0bf612-05ae-400f-9511-af776c601549-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1c0bf612-05ae-400f-9511-af776c601549" (UID: "1c0bf612-05ae-400f-9511-af776c601549"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 11:00:30.909162 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.909142    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c0bf612-05ae-400f-9511-af776c601549-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1c0bf612-05ae-400f-9511-af776c601549" (UID: "1c0bf612-05ae-400f-9511-af776c601549"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 11:00:30.909242 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:30.909169    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c0bf612-05ae-400f-9511-af776c601549-kube-api-access-dw7zr" (OuterVolumeSpecName: "kube-api-access-dw7zr") pod "1c0bf612-05ae-400f-9511-af776c601549" (UID: "1c0bf612-05ae-400f-9511-af776c601549"). InnerVolumeSpecName "kube-api-access-dw7zr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:00:31.007832 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:31.007804    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dw7zr\" (UniqueName: \"kubernetes.io/projected/1c0bf612-05ae-400f-9511-af776c601549-kube-api-access-dw7zr\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:00:31.007832 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:31.007828    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:00:31.008011 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:31.007842    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-console-config\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:00:31.008011 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:31.007853    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c0bf612-05ae-400f-9511-af776c601549-console-serving-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:00:31.008011 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:31.007864    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c0bf612-05ae-400f-9511-af776c601549-console-oauth-config\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:00:31.008011 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:31.007875    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c0bf612-05ae-400f-9511-af776c601549-trusted-ca-bundle\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:00:31.129728 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:31.129697    2564 generic.go:356] "Generic (PLEG): container finished" podID="1c0bf612-05ae-400f-9511-af776c601549" containerID="cd4156e9ba5b4bedfdfcb9d46bbbc818ffe21d93911bd62e56442c5198346ac6" exitCode=0
Sep 17 11:00:31.129843 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:31.129753    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d4c6d598d-wh4dm"
Sep 17 11:00:31.129843 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:31.129759    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d4c6d598d-wh4dm" event={"ID":"1c0bf612-05ae-400f-9511-af776c601549","Type":"ContainerDied","Data":"cd4156e9ba5b4bedfdfcb9d46bbbc818ffe21d93911bd62e56442c5198346ac6"}
Sep 17 11:00:31.129843 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:31.129812    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d4c6d598d-wh4dm" event={"ID":"1c0bf612-05ae-400f-9511-af776c601549","Type":"ContainerDied","Data":"bd9c7533f0172c901e7813e667933a027a1016dba9cc3a21f5a888a3d71f9b32"}
Sep 17 11:00:31.129843 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:31.129840    2564 scope.go:117] "RemoveContainer" containerID="cd4156e9ba5b4bedfdfcb9d46bbbc818ffe21d93911bd62e56442c5198346ac6"
Sep 17 11:00:31.139292 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:31.139275    2564 scope.go:117] "RemoveContainer" containerID="cd4156e9ba5b4bedfdfcb9d46bbbc818ffe21d93911bd62e56442c5198346ac6"
Sep 17 11:00:31.139573 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:00:31.139554    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd4156e9ba5b4bedfdfcb9d46bbbc818ffe21d93911bd62e56442c5198346ac6\": container with ID starting with cd4156e9ba5b4bedfdfcb9d46bbbc818ffe21d93911bd62e56442c5198346ac6 not found: ID does not exist" containerID="cd4156e9ba5b4bedfdfcb9d46bbbc818ffe21d93911bd62e56442c5198346ac6"
Sep 17 11:00:31.139624 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:31.139582    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd4156e9ba5b4bedfdfcb9d46bbbc818ffe21d93911bd62e56442c5198346ac6"} err="failed to get container status \"cd4156e9ba5b4bedfdfcb9d46bbbc818ffe21d93911bd62e56442c5198346ac6\": rpc error: code = NotFound desc = could not find container \"cd4156e9ba5b4bedfdfcb9d46bbbc818ffe21d93911bd62e56442c5198346ac6\": container with ID starting with cd4156e9ba5b4bedfdfcb9d46bbbc818ffe21d93911bd62e56442c5198346ac6 not found: ID does not exist"
Sep 17 11:00:31.157935 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:31.157879    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d4c6d598d-wh4dm"]
Sep 17 11:00:31.163731 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:31.163711    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d4c6d598d-wh4dm"]
Sep 17 11:00:33.141560 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:00:33.141529    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c0bf612-05ae-400f-9511-af776c601549" path="/var/lib/kubelet/pods/1c0bf612-05ae-400f-9511-af776c601549/volumes"
Sep 17 11:02:05.121872 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.121839    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"]
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122160    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eba0e36c-051e-4b97-b453-9305f67ff6e6" containerName="pull"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122172    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eba0e36c-051e-4b97-b453-9305f67ff6e6" containerName="pull"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122181    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9713c405-ff92-43ba-a406-778a848cd2e8" containerName="util"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122188    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9713c405-ff92-43ba-a406-778a848cd2e8" containerName="util"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122199    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eba0e36c-051e-4b97-b453-9305f67ff6e6" containerName="extract"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122204    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eba0e36c-051e-4b97-b453-9305f67ff6e6" containerName="extract"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122210    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d58430-ee5e-4f4a-9790-8cad5cde3596" containerName="pull"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122215    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d58430-ee5e-4f4a-9790-8cad5cde3596" containerName="pull"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122220    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eba0e36c-051e-4b97-b453-9305f67ff6e6" containerName="util"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122225    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eba0e36c-051e-4b97-b453-9305f67ff6e6" containerName="util"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122235    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f3afda8-4b97-4821-a1df-a2d8b585a2f2" containerName="util"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122239    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f3afda8-4b97-4821-a1df-a2d8b585a2f2" containerName="util"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122245    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f3afda8-4b97-4821-a1df-a2d8b585a2f2" containerName="extract"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122250    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f3afda8-4b97-4821-a1df-a2d8b585a2f2" containerName="extract"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122257    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9713c405-ff92-43ba-a406-778a848cd2e8" containerName="extract"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122262    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9713c405-ff92-43ba-a406-778a848cd2e8" containerName="extract"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122269    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d58430-ee5e-4f4a-9790-8cad5cde3596" containerName="util"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122274    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d58430-ee5e-4f4a-9790-8cad5cde3596" containerName="util"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122281    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f3afda8-4b97-4821-a1df-a2d8b585a2f2" containerName="pull"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122286    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f3afda8-4b97-4821-a1df-a2d8b585a2f2" containerName="pull"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122291    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c0bf612-05ae-400f-9511-af776c601549" containerName="console"
Sep 17 11:02:05.122288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122297    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c0bf612-05ae-400f-9511-af776c601549" containerName="console"
Sep 17 11:02:05.122944 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122304    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d58430-ee5e-4f4a-9790-8cad5cde3596" containerName="extract"
Sep 17 11:02:05.122944 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122309    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d58430-ee5e-4f4a-9790-8cad5cde3596" containerName="extract"
Sep 17 11:02:05.122944 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122319    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9713c405-ff92-43ba-a406-778a848cd2e8" containerName="pull"
Sep 17 11:02:05.122944 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122324    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9713c405-ff92-43ba-a406-778a848cd2e8" containerName="pull"
Sep 17 11:02:05.122944 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122370    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eba0e36c-051e-4b97-b453-9305f67ff6e6" containerName="extract"
Sep 17 11:02:05.122944 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122376    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c0bf612-05ae-400f-9511-af776c601549" containerName="console"
Sep 17 11:02:05.122944 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122384    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f3afda8-4b97-4821-a1df-a2d8b585a2f2" containerName="extract"
Sep 17 11:02:05.122944 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122390    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9713c405-ff92-43ba-a406-778a848cd2e8" containerName="extract"
Sep 17 11:02:05.122944 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.122396    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8d58430-ee5e-4f4a-9790-8cad5cde3596" containerName="extract"
Sep 17 11:02:05.125273 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.125252    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"
Sep 17 11:02:05.128809 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.128783    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-rklfc\""
Sep 17 11:02:05.129582 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.129566    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 17 11:02:05.146616 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.146587    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"]
Sep 17 11:02:05.149843 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.149813    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"
Sep 17 11:02:05.150649 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.150627    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"]
Sep 17 11:02:05.178649 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.178622    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"]
Sep 17 11:02:05.273156 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.273130    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02659011-0c89-4124-b5e9-6c682f290ce8-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-lxnqd\" (UID: \"02659011-0c89-4124-b5e9-6c682f290ce8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"
Sep 17 11:02:05.273306 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.273166    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mssk\" (UniqueName: \"kubernetes.io/projected/02659011-0c89-4124-b5e9-6c682f290ce8-kube-api-access-5mssk\") pod \"payload-processing-5b54b7fbb7-lxnqd\" (UID: \"02659011-0c89-4124-b5e9-6c682f290ce8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"
Sep 17 11:02:05.273306 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.273188    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f1dd653-d524-43fe-ad48-34da34e506e5-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-frfj9\" (UID: \"2f1dd653-d524-43fe-ad48-34da34e506e5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"
Sep 17 11:02:05.273306 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.273239    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm447\" (UniqueName: \"kubernetes.io/projected/2f1dd653-d524-43fe-ad48-34da34e506e5-kube-api-access-mm447\") pod \"payload-pre-processing-59fbc47bf8-frfj9\" (UID: \"2f1dd653-d524-43fe-ad48-34da34e506e5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"
Sep 17 11:02:05.374362 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.374289    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mssk\" (UniqueName: \"kubernetes.io/projected/02659011-0c89-4124-b5e9-6c682f290ce8-kube-api-access-5mssk\") pod \"payload-processing-5b54b7fbb7-lxnqd\" (UID: \"02659011-0c89-4124-b5e9-6c682f290ce8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"
Sep 17 11:02:05.374362 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.374319    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f1dd653-d524-43fe-ad48-34da34e506e5-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-frfj9\" (UID: \"2f1dd653-d524-43fe-ad48-34da34e506e5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"
Sep 17 11:02:05.374362 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.374349    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm447\" (UniqueName: \"kubernetes.io/projected/2f1dd653-d524-43fe-ad48-34da34e506e5-kube-api-access-mm447\") pod \"payload-pre-processing-59fbc47bf8-frfj9\" (UID: \"2f1dd653-d524-43fe-ad48-34da34e506e5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"
Sep 17 11:02:05.374593 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.374438    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02659011-0c89-4124-b5e9-6c682f290ce8-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-lxnqd\" (UID: \"02659011-0c89-4124-b5e9-6c682f290ce8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"
Sep 17 11:02:05.375291 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.375265    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02659011-0c89-4124-b5e9-6c682f290ce8-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-lxnqd\" (UID: \"02659011-0c89-4124-b5e9-6c682f290ce8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"
Sep 17 11:02:05.375377 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.375265    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f1dd653-d524-43fe-ad48-34da34e506e5-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-frfj9\" (UID: \"2f1dd653-d524-43fe-ad48-34da34e506e5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"
Sep 17 11:02:05.388961 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.388929    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mssk\" (UniqueName: \"kubernetes.io/projected/02659011-0c89-4124-b5e9-6c682f290ce8-kube-api-access-5mssk\") pod \"payload-processing-5b54b7fbb7-lxnqd\" (UID: \"02659011-0c89-4124-b5e9-6c682f290ce8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"
Sep 17 11:02:05.392540 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.392516    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm447\" (UniqueName: \"kubernetes.io/projected/2f1dd653-d524-43fe-ad48-34da34e506e5-kube-api-access-mm447\") pod \"payload-pre-processing-59fbc47bf8-frfj9\" (UID: \"2f1dd653-d524-43fe-ad48-34da34e506e5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"
Sep 17 11:02:05.434672 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.434647    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"
Sep 17 11:02:05.459714 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.459691    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"
Sep 17 11:02:05.732632 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.732607    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"]
Sep 17 11:02:05.734765 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:02:05.734739    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f1dd653_d524_43fe_ad48_34da34e506e5.slice/crio-b7d9e2880a9efc1edaa65503e25ce2203895bf53955743922dee4c59c2012b03 WatchSource:0}: Error finding container b7d9e2880a9efc1edaa65503e25ce2203895bf53955743922dee4c59c2012b03: Status 404 returned error can't find the container with id b7d9e2880a9efc1edaa65503e25ce2203895bf53955743922dee4c59c2012b03
Sep 17 11:02:05.768859 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:05.768840    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"]
Sep 17 11:02:05.770197 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:02:05.770180    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02659011_0c89_4124_b5e9_6c682f290ce8.slice/crio-aaa709be47bdb86cfdd45f927011dd14a7ca14df945ba11104954160cd695840 WatchSource:0}: Error finding container aaa709be47bdb86cfdd45f927011dd14a7ca14df945ba11104954160cd695840: Status 404 returned error can't find the container with id aaa709be47bdb86cfdd45f927011dd14a7ca14df945ba11104954160cd695840
Sep 17 11:02:06.587038 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:06.587000    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd" event={"ID":"02659011-0c89-4124-b5e9-6c682f290ce8","Type":"ContainerStarted","Data":"aaa709be47bdb86cfdd45f927011dd14a7ca14df945ba11104954160cd695840"}
Sep 17 11:02:06.588512 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:06.588478    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9" event={"ID":"2f1dd653-d524-43fe-ad48-34da34e506e5","Type":"ContainerStarted","Data":"b7d9e2880a9efc1edaa65503e25ce2203895bf53955743922dee4c59c2012b03"}
Sep 17 11:02:07.595997 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:07.595879    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd" event={"ID":"02659011-0c89-4124-b5e9-6c682f290ce8","Type":"ContainerStarted","Data":"cd233d454240bc7436f36235966f2c55304647fcc141ce0192fc3c590bf4dde9"}
Sep 17 11:02:07.596416 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:07.596081    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"
Sep 17 11:02:07.597560 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:07.597533    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9" event={"ID":"2f1dd653-d524-43fe-ad48-34da34e506e5","Type":"ContainerStarted","Data":"86303742fe6d835cd827242428cb1e3b1e98397d3da05bff83781ea23711f6c9"}
Sep 17 11:02:07.597655 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:07.597611    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"
Sep 17 11:02:07.633834 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:07.633789    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd" podStartSLOduration=1.094818045 podStartE2EDuration="2.633775377s" totalImagesPullingTime="1.538957332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:02:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:02:07.63319711 +0000 UTC m=+522.949560270" watchObservedRunningTime="2026-09-17 11:02:07.633775377 +0000 UTC m=+522.950138531"
Sep 17 11:02:07.678064 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:07.678011    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9" podStartSLOduration=1.103065506 podStartE2EDuration="2.677993792s" totalImagesPullingTime="1.574928286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:02:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:02:07.676403302 +0000 UTC m=+522.992766457" watchObservedRunningTime="2026-09-17 11:02:07.677993792 +0000 UTC m=+522.994356949"
Sep 17 11:02:18.605532 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:18.605454    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"
Sep 17 11:02:18.605532 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:18.605519    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"
Sep 17 11:02:31.015855 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.015825    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"]
Sep 17 11:02:31.018444 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.018422    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.021266 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.021244    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 17 11:02:31.021399 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.021279    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4xkgq\""
Sep 17 11:02:31.021526 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.021511    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 17 11:02:31.021648 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.021629    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 17 11:02:31.032756 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.032736    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"]
Sep 17 11:02:31.085074 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.085051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.085180 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.085079    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.085180 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.085102    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.085180 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.085119    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.085299 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.085202    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8tz4\" (UniqueName: \"kubernetes.io/projected/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-kube-api-access-l8tz4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.085299 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.085233    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.186488 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.186457    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.186634 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.186493    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.186634 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.186543    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8tz4\" (UniqueName: \"kubernetes.io/projected/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-kube-api-access-l8tz4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.186634 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.186592    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.186802 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.186680    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.186802 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.186705    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.186932 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.186866    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.187220 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.187196    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.187345 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.187290    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.189134 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.189113    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.189230 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.189194    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.195130 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.195110    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8tz4\" (UniqueName: \"kubernetes.io/projected/2adaf5ab-1fab-4a7c-80f2-efb0f198eef3-kube-api-access-l8tz4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwsp8\" (UID: \"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.328314 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.328245    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:31.599836 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.599811    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"]
Sep 17 11:02:31.601019 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:02:31.600996    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2adaf5ab_1fab_4a7c_80f2_efb0f198eef3.slice/crio-71b5cbc70697469561213034d85c60ff1e3dabe6fd455c130a5ff0eb714bfe4d WatchSource:0}: Error finding container 71b5cbc70697469561213034d85c60ff1e3dabe6fd455c130a5ff0eb714bfe4d: Status 404 returned error can't find the container with id 71b5cbc70697469561213034d85c60ff1e3dabe6fd455c130a5ff0eb714bfe4d
Sep 17 11:02:31.719443 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:31.719409    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8" event={"ID":"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3","Type":"ContainerStarted","Data":"71b5cbc70697469561213034d85c60ff1e3dabe6fd455c130a5ff0eb714bfe4d"}
Sep 17 11:02:32.725979 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:32.725950    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8" event={"ID":"2adaf5ab-1fab-4a7c-80f2-efb0f198eef3","Type":"ContainerStarted","Data":"d201243b25ae2943283356199a4484f6a3f9ad444fc731d62b448ca29193572e"}
Sep 17 11:02:32.745922 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:32.745866    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8" podStartSLOduration=1.767382609 podStartE2EDuration="2.745850901s" totalImagesPullingTime="978.468292ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:02:32.74365958 +0000 UTC m=+548.060022735" watchObservedRunningTime="2026-09-17 11:02:32.745850901 +0000 UTC m=+548.062214098"
Sep 17 11:02:41.329078 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:41.329041    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:41.329540 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:41.329112    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:41.341593 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:41.341561    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:02:41.782476 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:02:41.782449    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwsp8"
Sep 17 11:03:01.224686 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.224645    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"]
Sep 17 11:03:01.228639 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.228623    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.230829 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.230802    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 17 11:03:01.238146 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.238112    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"]
Sep 17 11:03:01.319696 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.319654    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.319818 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.319709    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.319871 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.319835    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.319871 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.319864    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wznrk\" (UniqueName: \"kubernetes.io/projected/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-kube-api-access-wznrk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.319982 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.319889    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.319982 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.319939    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.421329 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.421300    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.421494 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.421337    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wznrk\" (UniqueName: \"kubernetes.io/projected/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-kube-api-access-wznrk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.421494 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.421361    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.421494 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.421378    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.421494 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.421410    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.421494 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.421431    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.421748 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.421707    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.421850 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.421818    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.421850 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.421846    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.423802 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.423780    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.424080 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.424063    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.431044 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.431009    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wznrk\" (UniqueName: \"kubernetes.io/projected/c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4-kube-api-access-wznrk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh\" (UID: \"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.538710 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.538677    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:01.817107 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.817084    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"]
Sep 17 11:03:01.818687 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:03:01.818662    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1c5bd03_a7d0_4f08_b35e_f37c77bef3e4.slice/crio-88bab8899bee08242c5315f0c0ce4be4c3fa676f2359534fc547248d2bda948f WatchSource:0}: Error finding container 88bab8899bee08242c5315f0c0ce4be4c3fa676f2359534fc547248d2bda948f: Status 404 returned error can't find the container with id 88bab8899bee08242c5315f0c0ce4be4c3fa676f2359534fc547248d2bda948f
Sep 17 11:03:01.872516 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:01.872483    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh" event={"ID":"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4","Type":"ContainerStarted","Data":"88bab8899bee08242c5315f0c0ce4be4c3fa676f2359534fc547248d2bda948f"}
Sep 17 11:03:02.879968 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:02.879933    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh" event={"ID":"c1c5bd03-a7d0-4f08-b35e-f37c77bef3e4","Type":"ContainerStarted","Data":"4629769196f8dfbdf17b61144fc85b53f0a0c0272504076be5807f9c0c0b1f4d"}
Sep 17 11:03:02.905499 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:02.905444    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh" podStartSLOduration=1.706431953 podStartE2EDuration="1.905430413s" totalImagesPullingTime="198.99846ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:03:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:03:02.903503835 +0000 UTC m=+578.219866990" watchObservedRunningTime="2026-09-17 11:03:02.905430413 +0000 UTC m=+578.221793568"
Sep 17 11:03:11.538861 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:11.538829    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:11.539277 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:11.538889    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:11.551065 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:11.551042    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:03:11.937286 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:03:11.937210    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzrgzh"
Sep 17 11:04:28.418701 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.418666    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"]
Sep 17 11:04:28.419188 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.418851    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd" podUID="02659011-0c89-4124-b5e9-6c682f290ce8" containerName="payload-processing" containerID="cri-o://cd233d454240bc7436f36235966f2c55304647fcc141ce0192fc3c590bf4dde9" gracePeriod=30
Sep 17 11:04:28.508611 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.508578    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"]
Sep 17 11:04:28.508813 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.508771    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9" podUID="2f1dd653-d524-43fe-ad48-34da34e506e5" containerName="payload-pre-processing" containerID="cri-o://86303742fe6d835cd827242428cb1e3b1e98397d3da05bff83781ea23711f6c9" gracePeriod=30
Sep 17 11:04:28.603998 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.603957    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd" podUID="02659011-0c89-4124-b5e9-6c682f290ce8" containerName="payload-processing" probeResult="failure" output="dial tcp 10.132.0.38:9004: connect: connection refused"
Sep 17 11:04:28.604158 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.603956    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9" podUID="2f1dd653-d524-43fe-ad48-34da34e506e5" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.37:9004: connect: connection refused"
Sep 17 11:04:28.770280 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.770251    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"
Sep 17 11:04:28.777296 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.777276    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"
Sep 17 11:04:28.820892 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.820858    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mm447\" (UniqueName: \"kubernetes.io/projected/2f1dd653-d524-43fe-ad48-34da34e506e5-kube-api-access-mm447\") pod \"2f1dd653-d524-43fe-ad48-34da34e506e5\" (UID: \"2f1dd653-d524-43fe-ad48-34da34e506e5\") "
Sep 17 11:04:28.821087 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.820932    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f1dd653-d524-43fe-ad48-34da34e506e5-plugins-config-volume\") pod \"2f1dd653-d524-43fe-ad48-34da34e506e5\" (UID: \"2f1dd653-d524-43fe-ad48-34da34e506e5\") "
Sep 17 11:04:28.821087 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.820962    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02659011-0c89-4124-b5e9-6c682f290ce8-plugins-config-volume\") pod \"02659011-0c89-4124-b5e9-6c682f290ce8\" (UID: \"02659011-0c89-4124-b5e9-6c682f290ce8\") "
Sep 17 11:04:28.821087 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.820997    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mssk\" (UniqueName: \"kubernetes.io/projected/02659011-0c89-4124-b5e9-6c682f290ce8-kube-api-access-5mssk\") pod \"02659011-0c89-4124-b5e9-6c682f290ce8\" (UID: \"02659011-0c89-4124-b5e9-6c682f290ce8\") "
Sep 17 11:04:28.822746 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.822659    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f1dd653-d524-43fe-ad48-34da34e506e5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2f1dd653-d524-43fe-ad48-34da34e506e5" (UID: "2f1dd653-d524-43fe-ad48-34da34e506e5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:04:28.822746 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.822697    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02659011-0c89-4124-b5e9-6c682f290ce8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "02659011-0c89-4124-b5e9-6c682f290ce8" (UID: "02659011-0c89-4124-b5e9-6c682f290ce8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:04:28.825345 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.825317    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02659011-0c89-4124-b5e9-6c682f290ce8-kube-api-access-5mssk" (OuterVolumeSpecName: "kube-api-access-5mssk") pod "02659011-0c89-4124-b5e9-6c682f290ce8" (UID: "02659011-0c89-4124-b5e9-6c682f290ce8"). InnerVolumeSpecName "kube-api-access-5mssk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:04:28.825457 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.825438    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f1dd653-d524-43fe-ad48-34da34e506e5-kube-api-access-mm447" (OuterVolumeSpecName: "kube-api-access-mm447") pod "2f1dd653-d524-43fe-ad48-34da34e506e5" (UID: "2f1dd653-d524-43fe-ad48-34da34e506e5"). InnerVolumeSpecName "kube-api-access-mm447". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:04:28.922119 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.922038    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm447\" (UniqueName: \"kubernetes.io/projected/2f1dd653-d524-43fe-ad48-34da34e506e5-kube-api-access-mm447\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:04:28.922119 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.922063    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f1dd653-d524-43fe-ad48-34da34e506e5-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:04:28.922119 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.922072    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02659011-0c89-4124-b5e9-6c682f290ce8-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:04:28.922119 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:28.922082    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mssk\" (UniqueName: \"kubernetes.io/projected/02659011-0c89-4124-b5e9-6c682f290ce8-kube-api-access-5mssk\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:04:29.318528 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.318490    2564 generic.go:356] "Generic (PLEG): container finished" podID="02659011-0c89-4124-b5e9-6c682f290ce8" containerID="cd233d454240bc7436f36235966f2c55304647fcc141ce0192fc3c590bf4dde9" exitCode=0
Sep 17 11:04:29.318710 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.318539    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"
Sep 17 11:04:29.318710 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.318579    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd" event={"ID":"02659011-0c89-4124-b5e9-6c682f290ce8","Type":"ContainerDied","Data":"cd233d454240bc7436f36235966f2c55304647fcc141ce0192fc3c590bf4dde9"}
Sep 17 11:04:29.318710 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.318641    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-lxnqd" event={"ID":"02659011-0c89-4124-b5e9-6c682f290ce8","Type":"ContainerDied","Data":"aaa709be47bdb86cfdd45f927011dd14a7ca14df945ba11104954160cd695840"}
Sep 17 11:04:29.318710 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.318672    2564 scope.go:117] "RemoveContainer" containerID="cd233d454240bc7436f36235966f2c55304647fcc141ce0192fc3c590bf4dde9"
Sep 17 11:04:29.319887 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.319868    2564 generic.go:356] "Generic (PLEG): container finished" podID="2f1dd653-d524-43fe-ad48-34da34e506e5" containerID="86303742fe6d835cd827242428cb1e3b1e98397d3da05bff83781ea23711f6c9" exitCode=0
Sep 17 11:04:29.319978 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.319930    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9" event={"ID":"2f1dd653-d524-43fe-ad48-34da34e506e5","Type":"ContainerDied","Data":"86303742fe6d835cd827242428cb1e3b1e98397d3da05bff83781ea23711f6c9"}
Sep 17 11:04:29.320026 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.319986    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9" event={"ID":"2f1dd653-d524-43fe-ad48-34da34e506e5","Type":"ContainerDied","Data":"b7d9e2880a9efc1edaa65503e25ce2203895bf53955743922dee4c59c2012b03"}
Sep 17 11:04:29.320026 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.319936    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"
Sep 17 11:04:29.327351 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.327338    2564 scope.go:117] "RemoveContainer" containerID="cd233d454240bc7436f36235966f2c55304647fcc141ce0192fc3c590bf4dde9"
Sep 17 11:04:29.327633 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:04:29.327615    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd233d454240bc7436f36235966f2c55304647fcc141ce0192fc3c590bf4dde9\": container with ID starting with cd233d454240bc7436f36235966f2c55304647fcc141ce0192fc3c590bf4dde9 not found: ID does not exist" containerID="cd233d454240bc7436f36235966f2c55304647fcc141ce0192fc3c590bf4dde9"
Sep 17 11:04:29.327704 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.327645    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd233d454240bc7436f36235966f2c55304647fcc141ce0192fc3c590bf4dde9"} err="failed to get container status \"cd233d454240bc7436f36235966f2c55304647fcc141ce0192fc3c590bf4dde9\": rpc error: code = NotFound desc = could not find container \"cd233d454240bc7436f36235966f2c55304647fcc141ce0192fc3c590bf4dde9\": container with ID starting with cd233d454240bc7436f36235966f2c55304647fcc141ce0192fc3c590bf4dde9 not found: ID does not exist"
Sep 17 11:04:29.327704 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.327667    2564 scope.go:117] "RemoveContainer" containerID="86303742fe6d835cd827242428cb1e3b1e98397d3da05bff83781ea23711f6c9"
Sep 17 11:04:29.340774 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.340744    2564 scope.go:117] "RemoveContainer" containerID="86303742fe6d835cd827242428cb1e3b1e98397d3da05bff83781ea23711f6c9"
Sep 17 11:04:29.341188 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:04:29.341159    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86303742fe6d835cd827242428cb1e3b1e98397d3da05bff83781ea23711f6c9\": container with ID starting with 86303742fe6d835cd827242428cb1e3b1e98397d3da05bff83781ea23711f6c9 not found: ID does not exist" containerID="86303742fe6d835cd827242428cb1e3b1e98397d3da05bff83781ea23711f6c9"
Sep 17 11:04:29.341282 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.341216    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86303742fe6d835cd827242428cb1e3b1e98397d3da05bff83781ea23711f6c9"} err="failed to get container status \"86303742fe6d835cd827242428cb1e3b1e98397d3da05bff83781ea23711f6c9\": rpc error: code = NotFound desc = could not find container \"86303742fe6d835cd827242428cb1e3b1e98397d3da05bff83781ea23711f6c9\": container with ID starting with 86303742fe6d835cd827242428cb1e3b1e98397d3da05bff83781ea23711f6c9 not found: ID does not exist"
Sep 17 11:04:29.345208 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.345187    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"]
Sep 17 11:04:29.350582 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.350562    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-frfj9"]
Sep 17 11:04:29.371034 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.371009    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"]
Sep 17 11:04:29.374246 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:29.374228    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-lxnqd"]
Sep 17 11:04:31.142892 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:31.142854    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02659011-0c89-4124-b5e9-6c682f290ce8" path="/var/lib/kubelet/pods/02659011-0c89-4124-b5e9-6c682f290ce8/volumes"
Sep 17 11:04:31.143286 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:31.143245    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f1dd653-d524-43fe-ad48-34da34e506e5" path="/var/lib/kubelet/pods/2f1dd653-d524-43fe-ad48-34da34e506e5/volumes"
Sep 17 11:04:42.830877 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:42.830841    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-bff58c8cf-9hqn9"]
Sep 17 11:04:42.831340 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:42.831234    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f1dd653-d524-43fe-ad48-34da34e506e5" containerName="payload-pre-processing"
Sep 17 11:04:42.831340 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:42.831246    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f1dd653-d524-43fe-ad48-34da34e506e5" containerName="payload-pre-processing"
Sep 17 11:04:42.831340 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:42.831254    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02659011-0c89-4124-b5e9-6c682f290ce8" containerName="payload-processing"
Sep 17 11:04:42.831340 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:42.831261    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="02659011-0c89-4124-b5e9-6c682f290ce8" containerName="payload-processing"
Sep 17 11:04:42.831340 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:42.831318    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f1dd653-d524-43fe-ad48-34da34e506e5" containerName="payload-pre-processing"
Sep 17 11:04:42.831340 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:42.831327    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="02659011-0c89-4124-b5e9-6c682f290ce8" containerName="payload-processing"
Sep 17 11:04:42.836029 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:42.836010    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-bff58c8cf-9hqn9"
Sep 17 11:04:42.838329 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:42.838310    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Sep 17 11:04:42.838520 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:42.838508    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-cbjhk\""
Sep 17 11:04:42.842970 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:42.842948    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-bff58c8cf-9hqn9"]
Sep 17 11:04:42.939290 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:42.939256    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgswj\" (UniqueName: \"kubernetes.io/projected/2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84-kube-api-access-fgswj\") pod \"ai-gateway-controller-bff58c8cf-9hqn9\" (UID: \"2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84\") " pod="opendatahub/ai-gateway-controller-bff58c8cf-9hqn9"
Sep 17 11:04:43.039920 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:43.039862    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgswj\" (UniqueName: \"kubernetes.io/projected/2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84-kube-api-access-fgswj\") pod \"ai-gateway-controller-bff58c8cf-9hqn9\" (UID: \"2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84\") " pod="opendatahub/ai-gateway-controller-bff58c8cf-9hqn9"
Sep 17 11:04:43.052812 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:43.052779    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgswj\" (UniqueName: \"kubernetes.io/projected/2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84-kube-api-access-fgswj\") pod \"ai-gateway-controller-bff58c8cf-9hqn9\" (UID: \"2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84\") " pod="opendatahub/ai-gateway-controller-bff58c8cf-9hqn9"
Sep 17 11:04:43.147666 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:43.147585    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-bff58c8cf-9hqn9"
Sep 17 11:04:43.421100 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:43.421078    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-bff58c8cf-9hqn9"]
Sep 17 11:04:43.422861 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:04:43.422838    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a8e0a4a_29dc_4f2e_8d33_c4135ad5fe84.slice/crio-1dcce956f66f054cddf2774d5c99b115f3689137c89851fa87e9e3d542c4953a WatchSource:0}: Error finding container 1dcce956f66f054cddf2774d5c99b115f3689137c89851fa87e9e3d542c4953a: Status 404 returned error can't find the container with id 1dcce956f66f054cddf2774d5c99b115f3689137c89851fa87e9e3d542c4953a
Sep 17 11:04:43.424633 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:43.424614    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 11:04:44.405252 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:44.405208    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-bff58c8cf-9hqn9" event={"ID":"2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84","Type":"ContainerStarted","Data":"1dcce956f66f054cddf2774d5c99b115f3689137c89851fa87e9e3d542c4953a"}
Sep 17 11:04:46.060866 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.060834    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5d7dbbd56b-rl88q"]
Sep 17 11:04:46.064278 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.064262    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q"
Sep 17 11:04:46.071568 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.071549    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-rklfc\""
Sep 17 11:04:46.071692 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.071591    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 17 11:04:46.095854 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.095827    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5d7dbbd56b-rl88q"]
Sep 17 11:04:46.132291 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.132261    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg"]
Sep 17 11:04:46.135831 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.135813    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg"
Sep 17 11:04:46.156100 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.156073    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg"]
Sep 17 11:04:46.167058 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.167030    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29vgv\" (UniqueName: \"kubernetes.io/projected/995b3ba4-d76f-4b47-8194-940111359da7-kube-api-access-29vgv\") pod \"payload-processing-5d7dbbd56b-rl88q\" (UID: \"995b3ba4-d76f-4b47-8194-940111359da7\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q"
Sep 17 11:04:46.167189 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.167096    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/995b3ba4-d76f-4b47-8194-940111359da7-plugins-config-volume\") pod \"payload-processing-5d7dbbd56b-rl88q\" (UID: \"995b3ba4-d76f-4b47-8194-940111359da7\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q"
Sep 17 11:04:46.267838 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.267804    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8gqf\" (UniqueName: \"kubernetes.io/projected/3130c736-c680-4e0a-ada7-189fd19b3541-kube-api-access-t8gqf\") pod \"payload-pre-processing-6cd864bc95-4l7wg\" (UID: \"3130c736-c680-4e0a-ada7-189fd19b3541\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg"
Sep 17 11:04:46.268019 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.267860    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29vgv\" (UniqueName: \"kubernetes.io/projected/995b3ba4-d76f-4b47-8194-940111359da7-kube-api-access-29vgv\") pod \"payload-processing-5d7dbbd56b-rl88q\" (UID: \"995b3ba4-d76f-4b47-8194-940111359da7\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q"
Sep 17 11:04:46.268019 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.267962    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/995b3ba4-d76f-4b47-8194-940111359da7-plugins-config-volume\") pod \"payload-processing-5d7dbbd56b-rl88q\" (UID: \"995b3ba4-d76f-4b47-8194-940111359da7\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q"
Sep 17 11:04:46.268019 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.267982    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3130c736-c680-4e0a-ada7-189fd19b3541-plugins-config-volume\") pod \"payload-pre-processing-6cd864bc95-4l7wg\" (UID: \"3130c736-c680-4e0a-ada7-189fd19b3541\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg"
Sep 17 11:04:46.268709 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.268690    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/995b3ba4-d76f-4b47-8194-940111359da7-plugins-config-volume\") pod \"payload-processing-5d7dbbd56b-rl88q\" (UID: \"995b3ba4-d76f-4b47-8194-940111359da7\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q"
Sep 17 11:04:46.277168 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.277141    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29vgv\" (UniqueName: \"kubernetes.io/projected/995b3ba4-d76f-4b47-8194-940111359da7-kube-api-access-29vgv\") pod \"payload-processing-5d7dbbd56b-rl88q\" (UID: \"995b3ba4-d76f-4b47-8194-940111359da7\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q"
Sep 17 11:04:46.369049 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.368953    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3130c736-c680-4e0a-ada7-189fd19b3541-plugins-config-volume\") pod \"payload-pre-processing-6cd864bc95-4l7wg\" (UID: \"3130c736-c680-4e0a-ada7-189fd19b3541\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg"
Sep 17 11:04:46.369049 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.369004    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8gqf\" (UniqueName: \"kubernetes.io/projected/3130c736-c680-4e0a-ada7-189fd19b3541-kube-api-access-t8gqf\") pod \"payload-pre-processing-6cd864bc95-4l7wg\" (UID: \"3130c736-c680-4e0a-ada7-189fd19b3541\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg"
Sep 17 11:04:46.369779 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.369757    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3130c736-c680-4e0a-ada7-189fd19b3541-plugins-config-volume\") pod \"payload-pre-processing-6cd864bc95-4l7wg\" (UID: \"3130c736-c680-4e0a-ada7-189fd19b3541\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg"
Sep 17 11:04:46.373817 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.373797    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q"
Sep 17 11:04:46.377524 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.377500    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8gqf\" (UniqueName: \"kubernetes.io/projected/3130c736-c680-4e0a-ada7-189fd19b3541-kube-api-access-t8gqf\") pod \"payload-pre-processing-6cd864bc95-4l7wg\" (UID: \"3130c736-c680-4e0a-ada7-189fd19b3541\") " pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg"
Sep 17 11:04:46.419044 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.419012    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-bff58c8cf-9hqn9" event={"ID":"2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84","Type":"ContainerStarted","Data":"3066380351e9ddef81808dc74ffa25f68c66b16525a68222afd7ebd85af1d819"}
Sep 17 11:04:46.419196 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.419135    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-bff58c8cf-9hqn9"
Sep 17 11:04:46.443662 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.436884    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-bff58c8cf-9hqn9" podStartSLOduration=2.300379928 podStartE2EDuration="4.436855732s" totalImagesPullingTime="2.136475804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:04:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:04:46.43571787 +0000 UTC m=+681.752081028" watchObservedRunningTime="2026-09-17 11:04:46.436855732 +0000 UTC m=+681.753218888"
Sep 17 11:04:46.447342 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.447077    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg"
Sep 17 11:04:46.652098 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.652074    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5d7dbbd56b-rl88q"]
Sep 17 11:04:46.653727 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:04:46.653694    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod995b3ba4_d76f_4b47_8194_940111359da7.slice/crio-ccedb83462b626db70603be49ef1703fa152420556c8ad4b4914f32cb5a495e6 WatchSource:0}: Error finding container ccedb83462b626db70603be49ef1703fa152420556c8ad4b4914f32cb5a495e6: Status 404 returned error can't find the container with id ccedb83462b626db70603be49ef1703fa152420556c8ad4b4914f32cb5a495e6
Sep 17 11:04:46.724594 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:46.724569    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg"]
Sep 17 11:04:46.727343 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:04:46.727315    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3130c736_c680_4e0a_ada7_189fd19b3541.slice/crio-b2420f70076affe55850ec2c850bbc41c332f88fc6a619e42e3d25d0c4371761 WatchSource:0}: Error finding container b2420f70076affe55850ec2c850bbc41c332f88fc6a619e42e3d25d0c4371761: Status 404 returned error can't find the container with id b2420f70076affe55850ec2c850bbc41c332f88fc6a619e42e3d25d0c4371761
Sep 17 11:04:47.425278 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:47.425238    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" event={"ID":"3130c736-c680-4e0a-ada7-189fd19b3541","Type":"ContainerStarted","Data":"b2420f70076affe55850ec2c850bbc41c332f88fc6a619e42e3d25d0c4371761"}
Sep 17 11:04:47.426389 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:47.426355    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" event={"ID":"995b3ba4-d76f-4b47-8194-940111359da7","Type":"ContainerStarted","Data":"ccedb83462b626db70603be49ef1703fa152420556c8ad4b4914f32cb5a495e6"}
Sep 17 11:04:48.433150 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:48.433115    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" event={"ID":"3130c736-c680-4e0a-ada7-189fd19b3541","Type":"ContainerStarted","Data":"878a47f02f4fcb8c95f4055c93799fda45c6a356e205b4369f180bcf038fc82f"}
Sep 17 11:04:48.433682 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:48.433220    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg"
Sep 17 11:04:48.434669 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:48.434647    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" event={"ID":"995b3ba4-d76f-4b47-8194-940111359da7","Type":"ContainerStarted","Data":"a93eceee37cd80cf0cb09779e14c74e9a4c2555248f50b90e222389f092a43e3"}
Sep 17 11:04:48.434773 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:48.434743    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q"
Sep 17 11:04:48.451567 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:48.451477    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" podStartSLOduration=1.193558217 podStartE2EDuration="2.451460683s" totalImagesPullingTime="1.257902466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:04:48.450531891 +0000 UTC m=+683.766895047" watchObservedRunningTime="2026-09-17 11:04:48.451460683 +0000 UTC m=+683.767824239"
Sep 17 11:04:48.468949 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:48.468868    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" podStartSLOduration=1.135456114 podStartE2EDuration="2.468853559s" totalImagesPullingTime="1.333397445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:04:48.466890558 +0000 UTC m=+683.783253716" watchObservedRunningTime="2026-09-17 11:04:48.468853559 +0000 UTC m=+683.785216750"
Sep 17 11:04:57.429524 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:57.429495    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-bff58c8cf-9hqn9"
Sep 17 11:04:57.635558 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:57.632888    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-6ccb46fcc-7zd6v"]
Sep 17 11:04:57.638687 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:57.638660    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6ccb46fcc-7zd6v"
Sep 17 11:04:57.644054 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:57.644030    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6ccb46fcc-7zd6v"]
Sep 17 11:04:57.776634 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:57.776596    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zn8d\" (UniqueName: \"kubernetes.io/projected/e1599327-332a-4f62-908b-04c8edaed19e-kube-api-access-9zn8d\") pod \"ai-gateway-controller-6ccb46fcc-7zd6v\" (UID: \"e1599327-332a-4f62-908b-04c8edaed19e\") " pod="opendatahub/ai-gateway-controller-6ccb46fcc-7zd6v"
Sep 17 11:04:57.877510 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:57.877478    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zn8d\" (UniqueName: \"kubernetes.io/projected/e1599327-332a-4f62-908b-04c8edaed19e-kube-api-access-9zn8d\") pod \"ai-gateway-controller-6ccb46fcc-7zd6v\" (UID: \"e1599327-332a-4f62-908b-04c8edaed19e\") " pod="opendatahub/ai-gateway-controller-6ccb46fcc-7zd6v"
Sep 17 11:04:57.885970 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:57.885946    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zn8d\" (UniqueName: \"kubernetes.io/projected/e1599327-332a-4f62-908b-04c8edaed19e-kube-api-access-9zn8d\") pod \"ai-gateway-controller-6ccb46fcc-7zd6v\" (UID: \"e1599327-332a-4f62-908b-04c8edaed19e\") " pod="opendatahub/ai-gateway-controller-6ccb46fcc-7zd6v"
Sep 17 11:04:57.949956 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:57.949885    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6ccb46fcc-7zd6v"
Sep 17 11:04:58.225874 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:58.225849    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6ccb46fcc-7zd6v"]
Sep 17 11:04:58.227966 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:04:58.227940    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1599327_332a_4f62_908b_04c8edaed19e.slice/crio-5070bb03f5491deb4a53db82de77a986c5adfdb9b59e74995ed2e84220627ee7 WatchSource:0}: Error finding container 5070bb03f5491deb4a53db82de77a986c5adfdb9b59e74995ed2e84220627ee7: Status 404 returned error can't find the container with id 5070bb03f5491deb4a53db82de77a986c5adfdb9b59e74995ed2e84220627ee7
Sep 17 11:04:58.493172 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:58.493133    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6ccb46fcc-7zd6v" event={"ID":"e1599327-332a-4f62-908b-04c8edaed19e","Type":"ContainerStarted","Data":"5070bb03f5491deb4a53db82de77a986c5adfdb9b59e74995ed2e84220627ee7"}
Sep 17 11:04:59.441844 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:59.441818    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg"
Sep 17 11:04:59.442065 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:59.442052    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q"
Sep 17 11:04:59.442414 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:59.442393    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:04:59.442493 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:59.442478    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:04:59.501838 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:59.501796    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6ccb46fcc-7zd6v" event={"ID":"e1599327-332a-4f62-908b-04c8edaed19e","Type":"ContainerStarted","Data":"8a3e14a172565143f0b8112f8bd6a47ed737a0d9be3e46da3d4e29ad25769992"}
Sep 17 11:04:59.502219 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:59.501889    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-6ccb46fcc-7zd6v"
Sep 17 11:04:59.520312 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:04:59.520270    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-6ccb46fcc-7zd6v" podStartSLOduration=2.034409639 podStartE2EDuration="2.520252503s" totalImagesPullingTime="485.842864ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:04:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:04:59.517968903 +0000 UTC m=+694.834332056" watchObservedRunningTime="2026-09-17 11:04:59.520252503 +0000 UTC m=+694.836615658"
Sep 17 11:05:10.509364 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:10.509337    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-6ccb46fcc-7zd6v"
Sep 17 11:05:10.556545 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:10.556520    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-bff58c8cf-9hqn9"]
Sep 17 11:05:10.556804 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:10.556731    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-bff58c8cf-9hqn9" podUID="2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84" containerName="manager" containerID="cri-o://3066380351e9ddef81808dc74ffa25f68c66b16525a68222afd7ebd85af1d819" gracePeriod=30
Sep 17 11:05:10.807243 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:10.807220    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-bff58c8cf-9hqn9"
Sep 17 11:05:10.881080 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:10.881053    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgswj\" (UniqueName: \"kubernetes.io/projected/2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84-kube-api-access-fgswj\") pod \"2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84\" (UID: \"2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84\") "
Sep 17 11:05:10.883302 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:10.883275    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84-kube-api-access-fgswj" (OuterVolumeSpecName: "kube-api-access-fgswj") pod "2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84" (UID: "2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84"). InnerVolumeSpecName "kube-api-access-fgswj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:05:10.982307 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:10.982284    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgswj\" (UniqueName: \"kubernetes.io/projected/2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84-kube-api-access-fgswj\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:05:11.564015 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:11.563987    2564 generic.go:356] "Generic (PLEG): container finished" podID="2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84" containerID="3066380351e9ddef81808dc74ffa25f68c66b16525a68222afd7ebd85af1d819" exitCode=0
Sep 17 11:05:11.564399 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:11.564028    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-bff58c8cf-9hqn9"
Sep 17 11:05:11.564399 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:11.564078    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-bff58c8cf-9hqn9" event={"ID":"2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84","Type":"ContainerDied","Data":"3066380351e9ddef81808dc74ffa25f68c66b16525a68222afd7ebd85af1d819"}
Sep 17 11:05:11.564399 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:11.564133    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-bff58c8cf-9hqn9" event={"ID":"2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84","Type":"ContainerDied","Data":"1dcce956f66f054cddf2774d5c99b115f3689137c89851fa87e9e3d542c4953a"}
Sep 17 11:05:11.564399 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:11.564153    2564 scope.go:117] "RemoveContainer" containerID="3066380351e9ddef81808dc74ffa25f68c66b16525a68222afd7ebd85af1d819"
Sep 17 11:05:11.573039 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:11.573022    2564 scope.go:117] "RemoveContainer" containerID="3066380351e9ddef81808dc74ffa25f68c66b16525a68222afd7ebd85af1d819"
Sep 17 11:05:11.573331 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:05:11.573310    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3066380351e9ddef81808dc74ffa25f68c66b16525a68222afd7ebd85af1d819\": container with ID starting with 3066380351e9ddef81808dc74ffa25f68c66b16525a68222afd7ebd85af1d819 not found: ID does not exist" containerID="3066380351e9ddef81808dc74ffa25f68c66b16525a68222afd7ebd85af1d819"
Sep 17 11:05:11.573395 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:11.573339    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3066380351e9ddef81808dc74ffa25f68c66b16525a68222afd7ebd85af1d819"} err="failed to get container status \"3066380351e9ddef81808dc74ffa25f68c66b16525a68222afd7ebd85af1d819\": rpc error: code = NotFound desc = could not find container \"3066380351e9ddef81808dc74ffa25f68c66b16525a68222afd7ebd85af1d819\": container with ID starting with 3066380351e9ddef81808dc74ffa25f68c66b16525a68222afd7ebd85af1d819 not found: ID does not exist"
Sep 17 11:05:11.580086 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:11.580059    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-bff58c8cf-9hqn9"]
Sep 17 11:05:11.584232 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:11.584210    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-bff58c8cf-9hqn9"]
Sep 17 11:05:13.142310 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:13.142270    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84" path="/var/lib/kubelet/pods/2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84/volumes"
Sep 17 11:05:32.912966 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:32.912932    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"]
Sep 17 11:05:32.913347 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:32.913276    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84" containerName="manager"
Sep 17 11:05:32.913347 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:32.913287    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84" containerName="manager"
Sep 17 11:05:32.913347 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:32.913341    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a8e0a4a-29dc-4f2e-8d33-c4135ad5fe84" containerName="manager"
Sep 17 11:05:32.917587 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:32.917571    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:32.926139 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:05:32.926113    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" is forbidden: User \"system:node:ip-10-0-129-205.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-129-205.ec2.internal' and this object" podUID="1016253a-7a65-4c96-ab4a-4b4524d207ca" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:32.932665 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:05:32.932643    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"e2e-inf-a-79a70a-openshift-default-dockercfg-pbls2\" is forbidden: User \"system:node:ip-10-0-129-205.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-129-205.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-79a70a-openshift-default-dockercfg-pbls2\"" type="*v1.Secret"
Sep 17 11:05:32.953042 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:32.953022    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"]
Sep 17 11:05:33.066112 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.066076    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-token\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.066112 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.066119    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-workload-socket\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.066319 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.066134    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-podinfo\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.066319 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.066153    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-envoy\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.066319 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.066196    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-credential-socket\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.066319 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.066215    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1016253a-7a65-4c96-ab4a-4b4524d207ca-istiod-ca-cert\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.066319 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.066234    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-data\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.066490 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.066316    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-workload-certs\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.066490 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.066354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-782fq\" (UniqueName: \"kubernetes.io/projected/1016253a-7a65-4c96-ab4a-4b4524d207ca-kube-api-access-782fq\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.167141 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.167067    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-token\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.167141 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.167110    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-workload-socket\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.167141 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.167126    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-podinfo\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.167141 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.167143    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-envoy\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.167481 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.167171    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-credential-socket\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.167481 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.167195    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1016253a-7a65-4c96-ab4a-4b4524d207ca-istiod-ca-cert\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.167481 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.167213    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-data\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.167481 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.167278    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-workload-certs\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.167481 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.167344    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-782fq\" (UniqueName: \"kubernetes.io/projected/1016253a-7a65-4c96-ab4a-4b4524d207ca-kube-api-access-782fq\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.167764 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.167525    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-workload-socket\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.167764 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.167673    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-credential-socket\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.167764 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.167711    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-data\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.167764 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.167733    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-workload-certs\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.168094 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.168073    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1016253a-7a65-4c96-ab4a-4b4524d207ca-istiod-ca-cert\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.169723 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.169695    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-envoy\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.169986 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.169960    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-podinfo\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.188209 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.188186    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-token\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.202187 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.202166    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-782fq\" (UniqueName: \"kubernetes.io/projected/1016253a-7a65-4c96-ab4a-4b4524d207ca-kube-api-access-782fq\") pod \"e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") " pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:33.257121 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.257092    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"]
Sep 17 11:05:33.263473 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.263401    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.268149 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.268129    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-766cc3-openshift-default-dockercfg-pwjhv\""
Sep 17 11:05:33.296606 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.296585    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"]
Sep 17 11:05:33.369523 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.369490    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-token\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.369656 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.369531    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-workload-certs\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.369656 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.369551    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-data\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.369656 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.369592    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-envoy\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.369656 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.369632    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-workload-socket\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.369791 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.369672    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istiod-ca-cert\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.369791 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.369690    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb86x\" (UniqueName: \"kubernetes.io/projected/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-kube-api-access-xb86x\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.369791 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.369709    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-credential-socket\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.369791 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.369760    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-podinfo\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.470321 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.470232    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-podinfo\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.470321 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.470286    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-token\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.470321 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.470317    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-workload-certs\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.470582 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.470344    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-data\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.470582 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.470544    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-envoy\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.470693 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.470598    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-workload-socket\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.470693 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.470653    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istiod-ca-cert\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.470843 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.470693    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb86x\" (UniqueName: \"kubernetes.io/projected/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-kube-api-access-xb86x\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.470843 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.470728    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-credential-socket\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.470843 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.470793    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-workload-certs\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.471176 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.470849    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-data\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.471176 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.470996    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-workload-socket\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.471272 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.471251    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-credential-socket\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.471614 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.471589    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istiod-ca-cert\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.472803 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.472775    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-envoy\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.472803 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.472794    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-podinfo\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.485962 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.485942    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-token\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.493930 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.493887    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb86x\" (UniqueName: \"kubernetes.io/projected/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-kube-api-access-xb86x\") pod \"e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") " pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.575384 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.575345    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:33.862604 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.862484    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"]
Sep 17 11:05:33.865314 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:05:33.865284    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd3d0799_5524_4b05_9f2f_d0cd2a90bdac.slice/crio-bb30f6f5e84654076f145c5806537103b55c94f9e90ff7b714cbb9aa2a6601c7 WatchSource:0}: Error finding container bb30f6f5e84654076f145c5806537103b55c94f9e90ff7b714cbb9aa2a6601c7: Status 404 returned error can't find the container with id bb30f6f5e84654076f145c5806537103b55c94f9e90ff7b714cbb9aa2a6601c7
Sep 17 11:05:33.867608 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.867576    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:05:33.867703 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.867687    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:05:33.867746 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:33.867726    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:05:34.184875 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:34.184805    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-79a70a-openshift-default-dockercfg-pbls2\""
Sep 17 11:05:34.188928 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:34.188879    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:34.479625 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:34.479597    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"]
Sep 17 11:05:34.481476 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:05:34.481446    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1016253a_7a65_4c96_ab4a_4b4524d207ca.slice/crio-783f25bc58c566ab1cff7bf611c2351ee1a6e2c03706362f22d7a944be860886 WatchSource:0}: Error finding container 783f25bc58c566ab1cff7bf611c2351ee1a6e2c03706362f22d7a944be860886: Status 404 returned error can't find the container with id 783f25bc58c566ab1cff7bf611c2351ee1a6e2c03706362f22d7a944be860886
Sep 17 11:05:34.483827 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:34.483798    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:05:34.483910 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:34.483863    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:05:34.483957 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:34.483913    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:05:34.684601 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:34.684567    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv" event={"ID":"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac","Type":"ContainerStarted","Data":"7f2e9c6b9f928f45d81edc3c5a512bfbf0b1d1102be75a97a1bc487b0d2e51a7"}
Sep 17 11:05:34.684850 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:34.684621    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv" event={"ID":"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac","Type":"ContainerStarted","Data":"bb30f6f5e84654076f145c5806537103b55c94f9e90ff7b714cbb9aa2a6601c7"}
Sep 17 11:05:34.686443 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:34.686414    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68" event={"ID":"1016253a-7a65-4c96-ab4a-4b4524d207ca","Type":"ContainerStarted","Data":"8df248d75d6a04064c9fc2b4cdd091df928f665000ef4d8b6aa590b47b901f48"}
Sep 17 11:05:34.686567 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:34.686465    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68" event={"ID":"1016253a-7a65-4c96-ab4a-4b4524d207ca","Type":"ContainerStarted","Data":"783f25bc58c566ab1cff7bf611c2351ee1a6e2c03706362f22d7a944be860886"}
Sep 17 11:05:34.731635 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:34.731538    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv" podStartSLOduration=1.731523382 podStartE2EDuration="1.731523382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:05:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:05:34.73011821 +0000 UTC m=+730.046481366" watchObservedRunningTime="2026-09-17 11:05:34.731523382 +0000 UTC m=+730.047886537"
Sep 17 11:05:34.781927 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:34.781847    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68" podStartSLOduration=2.781828406 podStartE2EDuration="2.781828406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:05:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:05:34.780829901 +0000 UTC m=+730.097193106" watchObservedRunningTime="2026-09-17 11:05:34.781828406 +0000 UTC m=+730.098191562"
Sep 17 11:05:35.189746 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:35.189717    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:35.195233 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:35.195211    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:35.576551 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:35.576520    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:35.581870 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:35.581847    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:35.692036 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:35.692000    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:35.692203 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:35.692059    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:35.692675 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:35.692654    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:35.692776 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:35.692713    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:05:39.452623 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:39.452589    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"]
Sep 17 11:05:39.456105 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:39.456089    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"
Sep 17 11:05:39.458374 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:39.458350    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-d7b97f-dockercfg-fkj6f\""
Sep 17 11:05:39.458487 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:39.458380    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-d7b97f\""
Sep 17 11:05:39.469333 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:39.469307    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"]
Sep 17 11:05:39.528090 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:39.528054    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gx9w\" (UniqueName: \"kubernetes.io/projected/f30081de-2cd4-4b52-8379-8c4c5021477e-kube-api-access-2gx9w\") pod \"payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx\" (UID: \"f30081de-2cd4-4b52-8379-8c4c5021477e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"
Sep 17 11:05:39.528245 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:39.528119    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f30081de-2cd4-4b52-8379-8c4c5021477e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx\" (UID: \"f30081de-2cd4-4b52-8379-8c4c5021477e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"
Sep 17 11:05:39.629121 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:39.629082    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gx9w\" (UniqueName: \"kubernetes.io/projected/f30081de-2cd4-4b52-8379-8c4c5021477e-kube-api-access-2gx9w\") pod \"payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx\" (UID: \"f30081de-2cd4-4b52-8379-8c4c5021477e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"
Sep 17 11:05:39.629316 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:39.629186    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f30081de-2cd4-4b52-8379-8c4c5021477e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx\" (UID: \"f30081de-2cd4-4b52-8379-8c4c5021477e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"
Sep 17 11:05:39.629842 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:39.629824    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f30081de-2cd4-4b52-8379-8c4c5021477e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx\" (UID: \"f30081de-2cd4-4b52-8379-8c4c5021477e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"
Sep 17 11:05:39.672277 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:39.672251    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gx9w\" (UniqueName: \"kubernetes.io/projected/f30081de-2cd4-4b52-8379-8c4c5021477e-kube-api-access-2gx9w\") pod \"payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx\" (UID: \"f30081de-2cd4-4b52-8379-8c4c5021477e\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"
Sep 17 11:05:39.766975 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:39.766953    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"
Sep 17 11:05:40.057940 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:40.057839    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"]
Sep 17 11:05:40.061231 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:05:40.061205    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf30081de_2cd4_4b52_8379_8c4c5021477e.slice/crio-89601f2aac98151a2ccb44d58ae9db4f9cd0f9b507120e1cdd11257cc1c91cfb WatchSource:0}: Error finding container 89601f2aac98151a2ccb44d58ae9db4f9cd0f9b507120e1cdd11257cc1c91cfb: Status 404 returned error can't find the container with id 89601f2aac98151a2ccb44d58ae9db4f9cd0f9b507120e1cdd11257cc1c91cfb
Sep 17 11:05:40.725328 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:40.725296    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx" event={"ID":"f30081de-2cd4-4b52-8379-8c4c5021477e","Type":"ContainerStarted","Data":"8d7896d83273710fc73500c902035e1449df5abf49099129b1ab8062894172b7"}
Sep 17 11:05:40.725736 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:40.725346    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx" event={"ID":"f30081de-2cd4-4b52-8379-8c4c5021477e","Type":"ContainerStarted","Data":"89601f2aac98151a2ccb44d58ae9db4f9cd0f9b507120e1cdd11257cc1c91cfb"}
Sep 17 11:05:40.725736 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:40.725399    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"
Sep 17 11:05:40.747908 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:40.747870    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"]
Sep 17 11:05:40.751347 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:40.751331    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"
Sep 17 11:05:40.753624 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:40.753607    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-79a70a\""
Sep 17 11:05:40.764290 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:40.764250    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx" podStartSLOduration=1.76423318 podStartE2EDuration="1.76423318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:05:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:05:40.764147856 +0000 UTC m=+736.080511010" watchObservedRunningTime="2026-09-17 11:05:40.76423318 +0000 UTC m=+736.080596340"
Sep 17 11:05:40.768746 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:40.768715    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"]
Sep 17 11:05:40.840392 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:40.840363    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e159a98d-069b-4bf9-b8da-48e695d971f1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs\" (UID: \"e159a98d-069b-4bf9-b8da-48e695d971f1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"
Sep 17 11:05:40.840552 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:40.840398    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwwpw\" (UniqueName: \"kubernetes.io/projected/e159a98d-069b-4bf9-b8da-48e695d971f1-kube-api-access-hwwpw\") pod \"payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs\" (UID: \"e159a98d-069b-4bf9-b8da-48e695d971f1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"
Sep 17 11:05:40.941264 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:40.941228    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e159a98d-069b-4bf9-b8da-48e695d971f1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs\" (UID: \"e159a98d-069b-4bf9-b8da-48e695d971f1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"
Sep 17 11:05:40.941264 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:40.941262    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwwpw\" (UniqueName: \"kubernetes.io/projected/e159a98d-069b-4bf9-b8da-48e695d971f1-kube-api-access-hwwpw\") pod \"payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs\" (UID: \"e159a98d-069b-4bf9-b8da-48e695d971f1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"
Sep 17 11:05:40.942038 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:40.942011    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e159a98d-069b-4bf9-b8da-48e695d971f1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs\" (UID: \"e159a98d-069b-4bf9-b8da-48e695d971f1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"
Sep 17 11:05:40.950111 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:40.950089    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwwpw\" (UniqueName: \"kubernetes.io/projected/e159a98d-069b-4bf9-b8da-48e695d971f1-kube-api-access-hwwpw\") pod \"payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs\" (UID: \"e159a98d-069b-4bf9-b8da-48e695d971f1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"
Sep 17 11:05:41.063777 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:41.063728    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"
Sep 17 11:05:41.355474 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:41.355450    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"]
Sep 17 11:05:41.357137 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:05:41.357111    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode159a98d_069b_4bf9_b8da_48e695d971f1.slice/crio-77359dc3a0c349b63b0f97d524e1166262ef67725092035038a11815bc75353d WatchSource:0}: Error finding container 77359dc3a0c349b63b0f97d524e1166262ef67725092035038a11815bc75353d: Status 404 returned error can't find the container with id 77359dc3a0c349b63b0f97d524e1166262ef67725092035038a11815bc75353d
Sep 17 11:05:41.731821 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:41.731727    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs" event={"ID":"e159a98d-069b-4bf9-b8da-48e695d971f1","Type":"ContainerStarted","Data":"1f5aef5b11496a1eb52909a3fa8642a2a8eb6de2e019050d05474395a985a408"}
Sep 17 11:05:41.732256 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:41.731837    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"
Sep 17 11:05:41.732256 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:41.731864    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs" event={"ID":"e159a98d-069b-4bf9-b8da-48e695d971f1","Type":"ContainerStarted","Data":"77359dc3a0c349b63b0f97d524e1166262ef67725092035038a11815bc75353d"}
Sep 17 11:05:41.784051 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:41.783998    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs" podStartSLOduration=1.783982642 podStartE2EDuration="1.783982642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:05:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:05:41.781262235 +0000 UTC m=+737.097625389" watchObservedRunningTime="2026-09-17 11:05:41.783982642 +0000 UTC m=+737.100345823"
Sep 17 11:05:42.446460 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.446425    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"]
Sep 17 11:05:42.450043 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.450022    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"
Sep 17 11:05:42.452117 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.452094    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-766cc3\""
Sep 17 11:05:42.452242 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.452179    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-766cc3-dockercfg-mt7k8\""
Sep 17 11:05:42.465331 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.465307    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"]
Sep 17 11:05:42.490765 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.490742    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"]
Sep 17 11:05:42.494147 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.494127    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"
Sep 17 11:05:42.518533 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.518487    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"]
Sep 17 11:05:42.555444 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.555406    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9676\" (UniqueName: \"kubernetes.io/projected/b3730c4b-e89d-41e6-afb5-07a4465f218b-kube-api-access-n9676\") pod \"payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c\" (UID: \"b3730c4b-e89d-41e6-afb5-07a4465f218b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"
Sep 17 11:05:42.555796 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.555759    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3730c4b-e89d-41e6-afb5-07a4465f218b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c\" (UID: \"b3730c4b-e89d-41e6-afb5-07a4465f218b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"
Sep 17 11:05:42.656790 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.656761    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vj2t\" (UniqueName: \"kubernetes.io/projected/8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe-kube-api-access-6vj2t\") pod \"payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p\" (UID: \"8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"
Sep 17 11:05:42.656946 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.656796    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9676\" (UniqueName: \"kubernetes.io/projected/b3730c4b-e89d-41e6-afb5-07a4465f218b-kube-api-access-n9676\") pod \"payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c\" (UID: \"b3730c4b-e89d-41e6-afb5-07a4465f218b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"
Sep 17 11:05:42.656946 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.656842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3730c4b-e89d-41e6-afb5-07a4465f218b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c\" (UID: \"b3730c4b-e89d-41e6-afb5-07a4465f218b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"
Sep 17 11:05:42.656946 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.656862    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p\" (UID: \"8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"
Sep 17 11:05:42.657557 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.657537    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3730c4b-e89d-41e6-afb5-07a4465f218b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c\" (UID: \"b3730c4b-e89d-41e6-afb5-07a4465f218b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"
Sep 17 11:05:42.669705 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.669681    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9676\" (UniqueName: \"kubernetes.io/projected/b3730c4b-e89d-41e6-afb5-07a4465f218b-kube-api-access-n9676\") pod \"payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c\" (UID: \"b3730c4b-e89d-41e6-afb5-07a4465f218b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"
Sep 17 11:05:42.758021 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.757990    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vj2t\" (UniqueName: \"kubernetes.io/projected/8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe-kube-api-access-6vj2t\") pod \"payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p\" (UID: \"8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"
Sep 17 11:05:42.758334 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.758058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p\" (UID: \"8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"
Sep 17 11:05:42.758672 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.758654    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p\" (UID: \"8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"
Sep 17 11:05:42.759851 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.759834    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"
Sep 17 11:05:42.771773 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.771745    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vj2t\" (UniqueName: \"kubernetes.io/projected/8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe-kube-api-access-6vj2t\") pod \"payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p\" (UID: \"8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"
Sep 17 11:05:42.804390 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:42.804344    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"
Sep 17 11:05:43.045514 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:43.045412    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"]
Sep 17 11:05:43.049218 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:05:43.049194    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3730c4b_e89d_41e6_afb5_07a4465f218b.slice/crio-6539ec11e855c299b5f42719c9ccdf264f50319652dab7bcd8747297b29a9bee WatchSource:0}: Error finding container 6539ec11e855c299b5f42719c9ccdf264f50319652dab7bcd8747297b29a9bee: Status 404 returned error can't find the container with id 6539ec11e855c299b5f42719c9ccdf264f50319652dab7bcd8747297b29a9bee
Sep 17 11:05:43.100014 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:43.099973    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"]
Sep 17 11:05:43.102324 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:05:43.102297    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f3e730f_6ef6_42e8_8ec9_54e3c0fe00fe.slice/crio-64dbd3eab9f99b2460c97c79d8a069dd39d8889f9992805d6ccdd5c0d5c2a32d WatchSource:0}: Error finding container 64dbd3eab9f99b2460c97c79d8a069dd39d8889f9992805d6ccdd5c0d5c2a32d: Status 404 returned error can't find the container with id 64dbd3eab9f99b2460c97c79d8a069dd39d8889f9992805d6ccdd5c0d5c2a32d
Sep 17 11:05:43.746264 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:43.746229    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p" event={"ID":"8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe","Type":"ContainerStarted","Data":"5af5213b8d8e2608a89c3583e11d5400a47f7ce2c1852f88ff1d6b254d5814fe"}
Sep 17 11:05:43.746450 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:43.746291    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p" event={"ID":"8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe","Type":"ContainerStarted","Data":"64dbd3eab9f99b2460c97c79d8a069dd39d8889f9992805d6ccdd5c0d5c2a32d"}
Sep 17 11:05:43.746450 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:43.746344    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"
Sep 17 11:05:43.748016 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:43.747987    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c" event={"ID":"b3730c4b-e89d-41e6-afb5-07a4465f218b","Type":"ContainerStarted","Data":"8f915154046d4a522fffa6838ed8556be853a1176e6a6e0d48ac98613af402c0"}
Sep 17 11:05:43.748112 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:43.748037    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c" event={"ID":"b3730c4b-e89d-41e6-afb5-07a4465f218b","Type":"ContainerStarted","Data":"6539ec11e855c299b5f42719c9ccdf264f50319652dab7bcd8747297b29a9bee"}
Sep 17 11:05:43.748112 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:43.748080    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"
Sep 17 11:05:43.782566 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:43.782524    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p" podStartSLOduration=1.782510996 podStartE2EDuration="1.782510996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:05:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:05:43.780675892 +0000 UTC m=+739.097039050" watchObservedRunningTime="2026-09-17 11:05:43.782510996 +0000 UTC m=+739.098874151"
Sep 17 11:05:43.809491 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:43.809452    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c" podStartSLOduration=1.809439742 podStartE2EDuration="1.809439742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:05:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:05:43.807239811 +0000 UTC m=+739.123602980" watchObservedRunningTime="2026-09-17 11:05:43.809439742 +0000 UTC m=+739.125802896"
Sep 17 11:05:51.733070 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:51.733023    2564 patch_prober.go:28] interesting pod/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 17 11:05:51.733528 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:51.733131    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68" podUID="1016253a-7a65-4c96-ab4a-4b4524d207ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 17 11:05:51.734315 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:51.734287    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"
Sep 17 11:05:52.740695 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:52.740668    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"
Sep 17 11:05:52.803281 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:52.801837    2564 generic.go:356] "Generic (PLEG): container finished" podID="1016253a-7a65-4c96-ab4a-4b4524d207ca" containerID="8df248d75d6a04064c9fc2b4cdd091df928f665000ef4d8b6aa590b47b901f48" exitCode=0
Sep 17 11:05:52.803281 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:52.801930    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68" event={"ID":"1016253a-7a65-4c96-ab4a-4b4524d207ca","Type":"ContainerDied","Data":"8df248d75d6a04064c9fc2b4cdd091df928f665000ef4d8b6aa590b47b901f48"}
Sep 17 11:05:52.803281 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:52.802423    2564 scope.go:117] "RemoveContainer" containerID="8df248d75d6a04064c9fc2b4cdd091df928f665000ef4d8b6aa590b47b901f48"
Sep 17 11:05:52.803958 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:52.803686    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:05:52.803958 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:52.803773    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:05:52.803958 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:52.803822    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:05:53.814050 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:53.814008    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68" event={"ID":"1016253a-7a65-4c96-ab4a-4b4524d207ca","Type":"ContainerStarted","Data":"bd369411caff1e75d6eb94582cc55ee3ba76023e434380c6f8be149298f065fd"}
Sep 17 11:05:54.190180 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:54.190044    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:54.190718 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:54.190693    2564 patch_prober.go:28] interesting pod/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 17 11:05:54.190983 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:54.190957    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68" podUID="1016253a-7a65-4c96-ab4a-4b4524d207ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 17 11:05:54.755872 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:54.755840    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"
Sep 17 11:05:54.756096 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:54.755969    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"
Sep 17 11:05:55.189569 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:55.189469    2564 patch_prober.go:28] interesting pod/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 17 11:05:55.189569 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:55.189539    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68" podUID="1016253a-7a65-4c96-ab4a-4b4524d207ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 17 11:05:56.189526 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:56.189483    2564 patch_prober.go:28] interesting pod/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 17 11:05:56.189810 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:56.189549    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68" podUID="1016253a-7a65-4c96-ab4a-4b4524d207ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 17 11:05:57.189692 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:57.189659    2564 patch_prober.go:28] interesting pod/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 17 11:05:57.190005 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:57.189971    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68" podUID="1016253a-7a65-4c96-ab4a-4b4524d207ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 17 11:05:58.190079 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:58.190032    2564 patch_prober.go:28] interesting pod/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 17 11:05:58.190575 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:58.190103    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68" podUID="1016253a-7a65-4c96-ab4a-4b4524d207ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 17 11:05:59.193465 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:59.193429    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:59.193948 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:59.193818    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:05:59.194598 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:05:59.194577    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:06:01.898505 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:01.898405    2564 generic.go:356] "Generic (PLEG): container finished" podID="bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" containerID="7f2e9c6b9f928f45d81edc3c5a512bfbf0b1d1102be75a97a1bc487b0d2e51a7" exitCode=0
Sep 17 11:06:01.898505 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:01.898481    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv" event={"ID":"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac","Type":"ContainerDied","Data":"7f2e9c6b9f928f45d81edc3c5a512bfbf0b1d1102be75a97a1bc487b0d2e51a7"}
Sep 17 11:06:01.898968 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:01.898855    2564 scope.go:117] "RemoveContainer" containerID="7f2e9c6b9f928f45d81edc3c5a512bfbf0b1d1102be75a97a1bc487b0d2e51a7"
Sep 17 11:06:01.899944 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:01.899912    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:06:01.900060 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:01.899984    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:06:01.900060 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:01.900025    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:06:02.910942 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:02.910883    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv" event={"ID":"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac","Type":"ContainerStarted","Data":"8c11eee5f8056c0458f453bdb19300d118ab1d156386348b099ea8415f112718"}
Sep 17 11:06:03.575881 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:03.575849    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:06:03.577030 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:03.577009    2564 patch_prober.go:28] interesting pod/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 17 11:06:03.577105 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:03.577079    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv" podUID="bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 17 11:06:04.136950 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:04.136919    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:06:04.576600 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:04.576564    2564 patch_prober.go:28] interesting pod/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 17 11:06:04.576776 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:04.576618    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv" podUID="bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 17 11:06:05.575681 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:05.575645    2564 patch_prober.go:28] interesting pod/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 17 11:06:05.576113 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:05.575700    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv" podUID="bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 17 11:06:05.692619 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:05.692583    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:06:06.576160 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:06.576124    2564 patch_prober.go:28] interesting pod/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 17 11:06:06.576548 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:06.576180    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv" podUID="bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 17 11:06:07.576176 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:07.576142    2564 patch_prober.go:28] interesting pod/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 17 11:06:07.576575 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:07.576198    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv" podUID="bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 17 11:06:08.579489 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:08.579460    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:06:08.580533 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:08.580509    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:06:23.137751 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:23.137717    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:06:26.359349 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.359320    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"]
Sep 17 11:06:26.362807 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.362789    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.365772 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.365753    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-d7b97f\"/\"openshift-service-ca.crt\""
Sep 17 11:06:26.365882 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.365798    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-d7b97f\"/\"default-dockercfg-v7mwd\""
Sep 17 11:06:26.366174 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.366160    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-d7b97f\"/\"simulator-w0-d7b97f-kserve-self-signed-certs\""
Sep 17 11:06:26.366677 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.366665    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-d7b97f\"/\"kube-root-ca.crt\""
Sep 17 11:06:26.377965 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.377943    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"]
Sep 17 11:06:26.468161 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.468131    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-model-cache\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.468314 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.468166    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-tmp-dir\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.468314 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.468202    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ncss\" (UniqueName: \"kubernetes.io/projected/cd7090ce-fe70-462c-85bc-c5e4a15840e1-kube-api-access-6ncss\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.468443 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.468317    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-home\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.468443 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.468381    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-dshm\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.468443 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.468410    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd7090ce-fe70-462c-85bc-c5e4a15840e1-tls-certs\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.569080 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.569052    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-model-cache\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.569080 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.569080    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-tmp-dir\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.569286 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.569110    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ncss\" (UniqueName: \"kubernetes.io/projected/cd7090ce-fe70-462c-85bc-c5e4a15840e1-kube-api-access-6ncss\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.569286 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.569169    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-home\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.569286 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.569203    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-dshm\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.569286 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.569217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd7090ce-fe70-462c-85bc-c5e4a15840e1-tls-certs\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.569520 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.569497    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-model-cache\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.569628 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.569538    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-tmp-dir\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.569628 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.569559    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-home\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.571670 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.571643    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-dshm\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.571776 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.571726    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd7090ce-fe70-462c-85bc-c5e4a15840e1-tls-certs\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.579478 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.579424    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ncss\" (UniqueName: \"kubernetes.io/projected/cd7090ce-fe70-462c-85bc-c5e4a15840e1-kube-api-access-6ncss\") pod \"simulator-w0-d7b97f-kserve-7df974d554-gh98k\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") " pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.672405 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.672325    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:26.948485 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:26.948455    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"]
Sep 17 11:06:26.949606 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:06:26.949582    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd7090ce_fe70_462c_85bc_c5e4a15840e1.slice/crio-7b3a3d16eabf47007a1c7b7ea6ea4ef4fdc6eaca59300c07f3d05b2d28189355 WatchSource:0}: Error finding container 7b3a3d16eabf47007a1c7b7ea6ea4ef4fdc6eaca59300c07f3d05b2d28189355: Status 404 returned error can't find the container with id 7b3a3d16eabf47007a1c7b7ea6ea4ef4fdc6eaca59300c07f3d05b2d28189355
Sep 17 11:06:27.052825 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:27.052796    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k" event={"ID":"cd7090ce-fe70-462c-85bc-c5e4a15840e1","Type":"ContainerStarted","Data":"934996d524c6c785d5fd4a024bfe1135770c5ba2009e8579beee260dedf5528a"}
Sep 17 11:06:27.053011 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:27.052844    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k" event={"ID":"cd7090ce-fe70-462c-85bc-c5e4a15840e1","Type":"ContainerStarted","Data":"7b3a3d16eabf47007a1c7b7ea6ea4ef4fdc6eaca59300c07f3d05b2d28189355"}
Sep 17 11:06:27.072617 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:27.072570    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k" podStartSLOduration=1.072549082 podStartE2EDuration="1.072549082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:06:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:06:27.070727617 +0000 UTC m=+782.387090772" watchObservedRunningTime="2026-09-17 11:06:27.072549082 +0000 UTC m=+782.388912236"
Sep 17 11:06:36.672587 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:36.672551    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:36.673000 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:36.672605    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:36.684948 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:36.684925    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:06:37.120592 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:06:37.120560    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:07:22.136734 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:22.136700    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:07:32.838028 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:32.837998    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"]
Sep 17 11:07:32.841534 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:32.841513    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:32.843813 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:32.843794    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-766cc3\"/\"openshift-service-ca.crt\""
Sep 17 11:07:32.844449 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:32.844430    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-766cc3\"/\"premium-w3-766cc3-kserve-self-signed-certs\""
Sep 17 11:07:32.844571 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:32.844477    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-766cc3\"/\"kube-root-ca.crt\""
Sep 17 11:07:32.844705 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:32.844690    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-766cc3\"/\"default-dockercfg-kcxrj\""
Sep 17 11:07:32.852674 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:32.852651    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"]
Sep 17 11:07:32.955545 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:32.955513    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46prx\" (UniqueName: \"kubernetes.io/projected/e0365f11-a09b-4a19-be32-0323f7d0bd0e-kube-api-access-46prx\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:32.955706 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:32.955573    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0365f11-a09b-4a19-be32-0323f7d0bd0e-tls-certs\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:32.955706 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:32.955602    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-model-cache\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:32.955706 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:32.955629    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-tmp-dir\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:32.955706 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:32.955656    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-home\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:32.955706 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:32.955680    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-dshm\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:33.056241 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:33.056217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-model-cache\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:33.056393 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:33.056258    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-tmp-dir\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:33.056393 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:33.056285    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-home\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:33.056393 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:33.056310    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-dshm\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:33.056393 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:33.056364    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46prx\" (UniqueName: \"kubernetes.io/projected/e0365f11-a09b-4a19-be32-0323f7d0bd0e-kube-api-access-46prx\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:33.056584 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:33.056411    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0365f11-a09b-4a19-be32-0323f7d0bd0e-tls-certs\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:33.056734 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:33.056713    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-tmp-dir\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:33.056801 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:33.056739    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-home\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:33.056801 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:33.056710    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-model-cache\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:33.058785 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:33.058761    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-dshm\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:33.059194 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:33.059176    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0365f11-a09b-4a19-be32-0323f7d0bd0e-tls-certs\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:33.067126 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:33.067079    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46prx\" (UniqueName: \"kubernetes.io/projected/e0365f11-a09b-4a19-be32-0323f7d0bd0e-kube-api-access-46prx\") pod \"premium-w3-766cc3-kserve-774bfb45bf-spjzs\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") " pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:33.152118 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:33.152042    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:33.429049 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:33.429011    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"]
Sep 17 11:07:33.430947 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:07:33.430923    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0365f11_a09b_4a19_be32_0323f7d0bd0e.slice/crio-afacc939ab50630e871effa0221c3f3e38f6f87a1e8f8996a48c962b5d4b35d5 WatchSource:0}: Error finding container afacc939ab50630e871effa0221c3f3e38f6f87a1e8f8996a48c962b5d4b35d5: Status 404 returned error can't find the container with id afacc939ab50630e871effa0221c3f3e38f6f87a1e8f8996a48c962b5d4b35d5
Sep 17 11:07:34.428951 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:34.428889    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs" event={"ID":"e0365f11-a09b-4a19-be32-0323f7d0bd0e","Type":"ContainerStarted","Data":"9f6814ae764d11feb28a253b8b3ab5e1ac6358ed2871a235263deba6d38700ef"}
Sep 17 11:07:34.429375 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:34.428957    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs" event={"ID":"e0365f11-a09b-4a19-be32-0323f7d0bd0e","Type":"ContainerStarted","Data":"afacc939ab50630e871effa0221c3f3e38f6f87a1e8f8996a48c962b5d4b35d5"}
Sep 17 11:07:34.451953 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:34.451891    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs" podStartSLOduration=2.451876106 podStartE2EDuration="2.451876106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:07:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:07:34.451738712 +0000 UTC m=+849.768101867" watchObservedRunningTime="2026-09-17 11:07:34.451876106 +0000 UTC m=+849.768239260"
Sep 17 11:07:43.152468 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:43.152424    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:43.153016 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:43.152503    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:43.165161 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:43.165134    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:43.491978 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:43.491953    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:07:51.136957 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:07:51.136925    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:08:29.143693 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:08:29.143661    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:08:59.137507 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:08:59.137470    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:09:19.592851 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.592818    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"]
Sep 17 11:09:19.596103 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.596088    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.600634 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.600615    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-d7b97f\"/\"distinct-w0-d7b97f-kserve-self-signed-certs\""
Sep 17 11:09:19.627856 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.627827    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"]
Sep 17 11:09:19.713639 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.713607    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-tmp-dir\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.713789 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.713647    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-model-cache\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.713789 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.713668    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xppdg\" (UniqueName: \"kubernetes.io/projected/604b7a3a-b16d-43c1-ad03-71fff923c079-kube-api-access-xppdg\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.713789 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.713737    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-home\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.713922 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.713793    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-dshm\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.713922 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.713821    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/604b7a3a-b16d-43c1-ad03-71fff923c079-tls-certs\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.814666 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.814633    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-dshm\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.814851 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.814674    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/604b7a3a-b16d-43c1-ad03-71fff923c079-tls-certs\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.814851 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.814738    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-tmp-dir\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.814851 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.814763    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-model-cache\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.814851 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.814785    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xppdg\" (UniqueName: \"kubernetes.io/projected/604b7a3a-b16d-43c1-ad03-71fff923c079-kube-api-access-xppdg\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.814851 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.814825    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-home\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.815205 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.815175    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-tmp-dir\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.815265 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.815212    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-home\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.815265 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.815236    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-model-cache\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.817161 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.817141    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-dshm\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.817310 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.817293    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/604b7a3a-b16d-43c1-ad03-71fff923c079-tls-certs\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.828663 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.828642    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xppdg\" (UniqueName: \"kubernetes.io/projected/604b7a3a-b16d-43c1-ad03-71fff923c079-kube-api-access-xppdg\") pod \"distinct-w0-d7b97f-kserve-775459d6f4-6fccp\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") " pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:19.905409 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:19.905349    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:20.206228 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:20.202655    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"]
Sep 17 11:09:20.210202 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:09:20.210173    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod604b7a3a_b16d_43c1_ad03_71fff923c079.slice/crio-4ccdc0f1da9ea41ad0bde0bb4e7205e717bc5d40cebc3791acbea0acbf4a3a77 WatchSource:0}: Error finding container 4ccdc0f1da9ea41ad0bde0bb4e7205e717bc5d40cebc3791acbea0acbf4a3a77: Status 404 returned error can't find the container with id 4ccdc0f1da9ea41ad0bde0bb4e7205e717bc5d40cebc3791acbea0acbf4a3a77
Sep 17 11:09:21.016129 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:21.016091    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp" event={"ID":"604b7a3a-b16d-43c1-ad03-71fff923c079","Type":"ContainerStarted","Data":"2cf6e8bbd477b80845a6aea4670029c6589f96c1e30f6b9e8446a9bbc1fbe20f"}
Sep 17 11:09:21.016484 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:21.016140    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp" event={"ID":"604b7a3a-b16d-43c1-ad03-71fff923c079","Type":"ContainerStarted","Data":"4ccdc0f1da9ea41ad0bde0bb4e7205e717bc5d40cebc3791acbea0acbf4a3a77"}
Sep 17 11:09:21.069840 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:21.069792    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp" podStartSLOduration=2.069778412 podStartE2EDuration="2.069778412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:09:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:09:21.066438553 +0000 UTC m=+956.382801707" watchObservedRunningTime="2026-09-17 11:09:21.069778412 +0000 UTC m=+956.386141566"
Sep 17 11:09:29.905503 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:29.905468    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:29.905976 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:29.905518    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:29.923588 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:29.923562    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:30.078631 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:30.078600    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:09:40.482676 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.482642    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"]
Sep 17 11:09:40.486434 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.486414    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.488538 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.488520    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-dfe3d913-openshift-default-dockercfg-r6phc\""
Sep 17 11:09:40.504039 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.504017    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"]
Sep 17 11:09:40.598398 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.598362    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-istio-envoy\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.598565 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.598431    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-istio-data\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.598565 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.598465    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8380c71e-7700-4059-9266-8926a325650e-istiod-ca-cert\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.598565 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.598504    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8380c71e-7700-4059-9266-8926a325650e-istio-podinfo\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.598703 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.598584    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-workload-socket\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.598703 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.598644    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-workload-certs\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.598703 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.598676    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8380c71e-7700-4059-9266-8926a325650e-istio-token\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.598825 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.598722    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-credential-socket\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.598825 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.598809    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwjdj\" (UniqueName: \"kubernetes.io/projected/8380c71e-7700-4059-9266-8926a325650e-kube-api-access-lwjdj\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.700345 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.700310    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwjdj\" (UniqueName: \"kubernetes.io/projected/8380c71e-7700-4059-9266-8926a325650e-kube-api-access-lwjdj\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.700629 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.700368    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-istio-envoy\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.700629 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.700404    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-istio-data\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.700629 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.700426    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8380c71e-7700-4059-9266-8926a325650e-istiod-ca-cert\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.700629 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.700463    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8380c71e-7700-4059-9266-8926a325650e-istio-podinfo\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.700629 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.700509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-workload-socket\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.700629 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.700552    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-workload-certs\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.700629 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.700596    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8380c71e-7700-4059-9266-8926a325650e-istio-token\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.701069 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.700645    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-credential-socket\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.701069 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.700962    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-istio-data\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.701069 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.700990    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-workload-socket\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.701069 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.701039    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-workload-certs\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.701302 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.701100    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-credential-socket\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.701302 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.701281    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8380c71e-7700-4059-9266-8926a325650e-istiod-ca-cert\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.702866 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.702846    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-istio-envoy\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.703096 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.703078    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8380c71e-7700-4059-9266-8926a325650e-istio-podinfo\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.713706 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.713653    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8380c71e-7700-4059-9266-8926a325650e-istio-token\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.714463 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.714417    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwjdj\" (UniqueName: \"kubernetes.io/projected/8380c71e-7700-4059-9266-8926a325650e-kube-api-access-lwjdj\") pod \"e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") " pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:40.796764 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:40.796735    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:41.083236 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:41.083210    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"]
Sep 17 11:09:41.085724 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:09:41.085697    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8380c71e_7700_4059_9266_8926a325650e.slice/crio-d54c377b39dbd38db48dbce85e633b90efac66587598d2485508a0ec1058137c WatchSource:0}: Error finding container d54c377b39dbd38db48dbce85e633b90efac66587598d2485508a0ec1058137c: Status 404 returned error can't find the container with id d54c377b39dbd38db48dbce85e633b90efac66587598d2485508a0ec1058137c
Sep 17 11:09:41.088132 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:41.088098    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:09:41.088217 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:41.088173    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:09:41.088217 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:41.088203    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:09:41.130353 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:41.130332    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq" event={"ID":"8380c71e-7700-4059-9266-8926a325650e","Type":"ContainerStarted","Data":"d54c377b39dbd38db48dbce85e633b90efac66587598d2485508a0ec1058137c"}
Sep 17 11:09:42.137283 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:42.137253    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq" event={"ID":"8380c71e-7700-4059-9266-8926a325650e","Type":"ContainerStarted","Data":"75ede7068abd4044910db2d480c79476b5249cecbfbf4908822f74ea1e135d27"}
Sep 17 11:09:42.159425 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:42.159376    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq" podStartSLOduration=2.159355345 podStartE2EDuration="2.159355345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:09:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:09:42.157615328 +0000 UTC m=+977.473978492" watchObservedRunningTime="2026-09-17 11:09:42.159355345 +0000 UTC m=+977.475718501"
Sep 17 11:09:42.797786 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:42.797752    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:42.802992 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:42.802969    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:43.143508 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:43.143419    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:43.144173 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:43.144155    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:09:47.866642 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:47.866565    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"]
Sep 17 11:09:47.870370 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:47.870349    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"
Sep 17 11:09:47.872652 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:47.872628    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-dfe3d913\""
Sep 17 11:09:47.872763 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:47.872675    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-dfe3d913-dockercfg-q5wdv\""
Sep 17 11:09:47.874469 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:47.874450    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea7c5878-2b13-400c-bd90-c2ae2b11962c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4\" (UID: \"ea7c5878-2b13-400c-bd90-c2ae2b11962c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"
Sep 17 11:09:47.874560 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:47.874481    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgmbq\" (UniqueName: \"kubernetes.io/projected/ea7c5878-2b13-400c-bd90-c2ae2b11962c-kube-api-access-tgmbq\") pod \"payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4\" (UID: \"ea7c5878-2b13-400c-bd90-c2ae2b11962c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"
Sep 17 11:09:47.887985 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:47.887960    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"]
Sep 17 11:09:47.891407 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:47.891388    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"]
Sep 17 11:09:47.891532 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:47.891467    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"
Sep 17 11:09:47.903563 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:47.903539    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"]
Sep 17 11:09:47.975060 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:47.975017    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea7c5878-2b13-400c-bd90-c2ae2b11962c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4\" (UID: \"ea7c5878-2b13-400c-bd90-c2ae2b11962c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"
Sep 17 11:09:47.975247 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:47.975091    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgmbq\" (UniqueName: \"kubernetes.io/projected/ea7c5878-2b13-400c-bd90-c2ae2b11962c-kube-api-access-tgmbq\") pod \"payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4\" (UID: \"ea7c5878-2b13-400c-bd90-c2ae2b11962c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"
Sep 17 11:09:47.975247 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:47.975158    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c871b3a0-4a8f-4113-9398-7af8c52b5a14-plugins-config-volume\") pod \"payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6\" (UID: \"c871b3a0-4a8f-4113-9398-7af8c52b5a14\") " pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"
Sep 17 11:09:47.975247 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:47.975193    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pnjx\" (UniqueName: \"kubernetes.io/projected/c871b3a0-4a8f-4113-9398-7af8c52b5a14-kube-api-access-5pnjx\") pod \"payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6\" (UID: \"c871b3a0-4a8f-4113-9398-7af8c52b5a14\") " pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"
Sep 17 11:09:47.975953 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:47.975881    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea7c5878-2b13-400c-bd90-c2ae2b11962c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4\" (UID: \"ea7c5878-2b13-400c-bd90-c2ae2b11962c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"
Sep 17 11:09:47.985450 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:47.985398    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgmbq\" (UniqueName: \"kubernetes.io/projected/ea7c5878-2b13-400c-bd90-c2ae2b11962c-kube-api-access-tgmbq\") pod \"payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4\" (UID: \"ea7c5878-2b13-400c-bd90-c2ae2b11962c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"
Sep 17 11:09:48.076359 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:48.076321    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pnjx\" (UniqueName: \"kubernetes.io/projected/c871b3a0-4a8f-4113-9398-7af8c52b5a14-kube-api-access-5pnjx\") pod \"payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6\" (UID: \"c871b3a0-4a8f-4113-9398-7af8c52b5a14\") " pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"
Sep 17 11:09:48.076571 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:48.076464    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c871b3a0-4a8f-4113-9398-7af8c52b5a14-plugins-config-volume\") pod \"payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6\" (UID: \"c871b3a0-4a8f-4113-9398-7af8c52b5a14\") " pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"
Sep 17 11:09:48.077171 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:48.077152    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c871b3a0-4a8f-4113-9398-7af8c52b5a14-plugins-config-volume\") pod \"payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6\" (UID: \"c871b3a0-4a8f-4113-9398-7af8c52b5a14\") " pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"
Sep 17 11:09:48.090677 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:48.090654    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pnjx\" (UniqueName: \"kubernetes.io/projected/c871b3a0-4a8f-4113-9398-7af8c52b5a14-kube-api-access-5pnjx\") pod \"payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6\" (UID: \"c871b3a0-4a8f-4113-9398-7af8c52b5a14\") " pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"
Sep 17 11:09:48.180938 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:48.180849    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"
Sep 17 11:09:48.202078 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:48.202039    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"
Sep 17 11:09:48.459610 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:48.459569    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"]
Sep 17 11:09:48.461089 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:09:48.461062    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea7c5878_2b13_400c_bd90_c2ae2b11962c.slice/crio-05f9b7771a6b294fba59ac92a1d039782c98748f4b88db560cba5a1892207363 WatchSource:0}: Error finding container 05f9b7771a6b294fba59ac92a1d039782c98748f4b88db560cba5a1892207363: Status 404 returned error can't find the container with id 05f9b7771a6b294fba59ac92a1d039782c98748f4b88db560cba5a1892207363
Sep 17 11:09:48.489240 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:48.489218    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"]
Sep 17 11:09:48.494027 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:09:48.494005    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc871b3a0_4a8f_4113_9398_7af8c52b5a14.slice/crio-3bf263127782b8ac003951e354f6448e23082b47f9b23d54b58a4dae66dfacba WatchSource:0}: Error finding container 3bf263127782b8ac003951e354f6448e23082b47f9b23d54b58a4dae66dfacba: Status 404 returned error can't find the container with id 3bf263127782b8ac003951e354f6448e23082b47f9b23d54b58a4dae66dfacba
Sep 17 11:09:49.181350 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:49.181314    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4" event={"ID":"ea7c5878-2b13-400c-bd90-c2ae2b11962c","Type":"ContainerStarted","Data":"d4326d46c221d98ed44adefc275c64729d3ef671a638164b23a4b498c5fba621"}
Sep 17 11:09:49.181815 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:49.181480    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"
Sep 17 11:09:49.181815 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:49.181522    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4" event={"ID":"ea7c5878-2b13-400c-bd90-c2ae2b11962c","Type":"ContainerStarted","Data":"05f9b7771a6b294fba59ac92a1d039782c98748f4b88db560cba5a1892207363"}
Sep 17 11:09:49.183053 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:49.183029    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6" event={"ID":"c871b3a0-4a8f-4113-9398-7af8c52b5a14","Type":"ContainerStarted","Data":"11086a23384c23573d0c73b8e27dbf90ac47cd6dfde7c0d544c75abdf0a88429"}
Sep 17 11:09:49.183193 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:49.183064    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6" event={"ID":"c871b3a0-4a8f-4113-9398-7af8c52b5a14","Type":"ContainerStarted","Data":"3bf263127782b8ac003951e354f6448e23082b47f9b23d54b58a4dae66dfacba"}
Sep 17 11:09:49.183193 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:49.183163    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"
Sep 17 11:09:49.200506 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:49.200466    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4" podStartSLOduration=2.200449264 podStartE2EDuration="2.200449264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:09:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:09:49.197953686 +0000 UTC m=+984.514316840" watchObservedRunningTime="2026-09-17 11:09:49.200449264 +0000 UTC m=+984.516812418"
Sep 17 11:09:49.223406 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:49.223360    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6" podStartSLOduration=2.223340925 podStartE2EDuration="2.223340925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:09:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:09:49.219035647 +0000 UTC m=+984.535398805" watchObservedRunningTime="2026-09-17 11:09:49.223340925 +0000 UTC m=+984.539704084"
Sep 17 11:09:56.137488 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:09:56.137448    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:10:00.136836 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:00.136802    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:10:00.190667 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:00.190637    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"
Sep 17 11:10:00.190820 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:00.190699    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"
Sep 17 11:10:12.317264 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:12.317233    2564 generic.go:356] "Generic (PLEG): container finished" podID="8380c71e-7700-4059-9266-8926a325650e" containerID="75ede7068abd4044910db2d480c79476b5249cecbfbf4908822f74ea1e135d27" exitCode=0
Sep 17 11:10:12.317664 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:12.317311    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq" event={"ID":"8380c71e-7700-4059-9266-8926a325650e","Type":"ContainerDied","Data":"75ede7068abd4044910db2d480c79476b5249cecbfbf4908822f74ea1e135d27"}
Sep 17 11:10:12.317748 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:12.317733    2564 scope.go:117] "RemoveContainer" containerID="75ede7068abd4044910db2d480c79476b5249cecbfbf4908822f74ea1e135d27"
Sep 17 11:10:12.319001 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:12.318968    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:10:12.319074 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:12.319044    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:10:12.319111 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:12.319076    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:10:13.143943 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:13.143889    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:10:13.324506 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:13.324469    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq" event={"ID":"8380c71e-7700-4059-9266-8926a325650e","Type":"ContainerStarted","Data":"657e0a84f744a8e1cc8eefaa364bde3eb63608560d3060c3bda237044520db24"}
Sep 17 11:10:13.798141 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:13.798108    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:10:13.799516 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:13.799487    2564 patch_prober.go:28] interesting pod/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 17 11:10:13.799753 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:13.799715    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq" podUID="8380c71e-7700-4059-9266-8926a325650e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 17 11:10:14.257210 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:14.257171    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd"]
Sep 17 11:10:14.260069 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:14.260047    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd"
Sep 17 11:10:14.262206 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:14.262187    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:10:14.262302 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:14.262228    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:10:14.268751 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:14.268727    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd"]
Sep 17 11:10:14.332460 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:14.332428    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80e29cb4-f401-4f8f-99e4-50c481ad8acf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd\" (UID: \"80e29cb4-f401-4f8f-99e4-50c481ad8acf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd"
Sep 17 11:10:14.433301 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:14.433266    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80e29cb4-f401-4f8f-99e4-50c481ad8acf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd\" (UID: \"80e29cb4-f401-4f8f-99e4-50c481ad8acf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd"
Sep 17 11:10:14.433842 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:14.433822    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80e29cb4-f401-4f8f-99e4-50c481ad8acf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd\" (UID: \"80e29cb4-f401-4f8f-99e4-50c481ad8acf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd"
Sep 17 11:10:14.571861 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:14.571781    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd"
Sep 17 11:10:14.801687 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:14.801235    2564 patch_prober.go:28] interesting pod/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 17 11:10:14.801687 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:14.801294    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq" podUID="8380c71e-7700-4059-9266-8926a325650e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 17 11:10:14.872702 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:14.872675    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd"]
Sep 17 11:10:14.874294 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:10:14.874265    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80e29cb4_f401_4f8f_99e4_50c481ad8acf.slice/crio-9e5a9874f3af0c89016ac0ddf940a5cc249a1192963372fab40a87982977d081 WatchSource:0}: Error finding container 9e5a9874f3af0c89016ac0ddf940a5cc249a1192963372fab40a87982977d081: Status 404 returned error can't find the container with id 9e5a9874f3af0c89016ac0ddf940a5cc249a1192963372fab40a87982977d081
Sep 17 11:10:14.876626 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:14.876610    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 11:10:15.337800 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:15.337760    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd" event={"ID":"80e29cb4-f401-4f8f-99e4-50c481ad8acf","Type":"ContainerStarted","Data":"9e5a9874f3af0c89016ac0ddf940a5cc249a1192963372fab40a87982977d081"}
Sep 17 11:10:15.806925 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:15.805143    2564 patch_prober.go:28] interesting pod/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 17 11:10:15.806925 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:15.805240    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq" podUID="8380c71e-7700-4059-9266-8926a325650e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 17 11:10:16.801711 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:16.801266    2564 patch_prober.go:28] interesting pod/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 17 11:10:16.801711 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:16.801329    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq" podUID="8380c71e-7700-4059-9266-8926a325650e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 17 11:10:17.797421 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:17.797384    2564 patch_prober.go:28] interesting pod/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 17 11:10:17.797583 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:17.797456    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq" podUID="8380c71e-7700-4059-9266-8926a325650e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 17 11:10:18.358819 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:18.358785    2564 generic.go:356] "Generic (PLEG): container finished" podID="80e29cb4-f401-4f8f-99e4-50c481ad8acf" containerID="0ffe32164ad59d78e3fea25ac07d73b8bdb481d62ceacb2055e3f16ea91a1f5d" exitCode=0
Sep 17 11:10:18.359163 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:18.358870    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd" event={"ID":"80e29cb4-f401-4f8f-99e4-50c481ad8acf","Type":"ContainerDied","Data":"0ffe32164ad59d78e3fea25ac07d73b8bdb481d62ceacb2055e3f16ea91a1f5d"}
Sep 17 11:10:18.797879 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:18.797838    2564 patch_prober.go:28] interesting pod/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 17 11:10:18.798208 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:18.798175    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq" podUID="8380c71e-7700-4059-9266-8926a325650e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 17 11:10:19.501752 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:19.501729    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd"
Sep 17 11:10:19.588636 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:19.588607    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80e29cb4-f401-4f8f-99e4-50c481ad8acf-maas-api-service-ca\") pod \"80e29cb4-f401-4f8f-99e4-50c481ad8acf\" (UID: \"80e29cb4-f401-4f8f-99e4-50c481ad8acf\") "
Sep 17 11:10:19.589001 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:19.588977    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80e29cb4-f401-4f8f-99e4-50c481ad8acf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "80e29cb4-f401-4f8f-99e4-50c481ad8acf" (UID: "80e29cb4-f401-4f8f-99e4-50c481ad8acf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:10:19.689509 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:19.689444    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80e29cb4-f401-4f8f-99e4-50c481ad8acf-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:10:19.802008 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:19.801966    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:10:19.802389 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:19.802367    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:10:19.803065 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:19.803046    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:10:20.371284 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:20.371255    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd"
Sep 17 11:10:20.371473 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:20.371291    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd" event={"ID":"80e29cb4-f401-4f8f-99e4-50c481ad8acf","Type":"ContainerDied","Data":"9e5a9874f3af0c89016ac0ddf940a5cc249a1192963372fab40a87982977d081"}
Sep 17 11:10:20.371473 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:20.371339    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e5a9874f3af0c89016ac0ddf940a5cc249a1192963372fab40a87982977d081"
Sep 17 11:10:25.689203 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:25.689169    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"]
Sep 17 11:10:25.689759 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:25.689541    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4" podUID="ea7c5878-2b13-400c-bd90-c2ae2b11962c" containerName="payload-pre-processing" containerID="cri-o://d4326d46c221d98ed44adefc275c64729d3ef671a638164b23a4b498c5fba621" gracePeriod=30
Sep 17 11:10:25.697063 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:25.697042    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"]
Sep 17 11:10:25.697249 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:25.697227    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6" podUID="c871b3a0-4a8f-4113-9398-7af8c52b5a14" containerName="payload-processing" containerID="cri-o://11086a23384c23573d0c73b8e27dbf90ac47cd6dfde7c0d544c75abdf0a88429" gracePeriod=30
Sep 17 11:10:26.005794 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.005766    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"
Sep 17 11:10:26.009235 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.009213    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"
Sep 17 11:10:26.060157 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.060120    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd"]
Sep 17 11:10:26.070973 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.070937    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dfe3d913-wsvvd"]
Sep 17 11:10:26.155242 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.155212    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgmbq\" (UniqueName: \"kubernetes.io/projected/ea7c5878-2b13-400c-bd90-c2ae2b11962c-kube-api-access-tgmbq\") pod \"ea7c5878-2b13-400c-bd90-c2ae2b11962c\" (UID: \"ea7c5878-2b13-400c-bd90-c2ae2b11962c\") "
Sep 17 11:10:26.155431 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.155257    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea7c5878-2b13-400c-bd90-c2ae2b11962c-plugins-config-volume\") pod \"ea7c5878-2b13-400c-bd90-c2ae2b11962c\" (UID: \"ea7c5878-2b13-400c-bd90-c2ae2b11962c\") "
Sep 17 11:10:26.155431 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.155325    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pnjx\" (UniqueName: \"kubernetes.io/projected/c871b3a0-4a8f-4113-9398-7af8c52b5a14-kube-api-access-5pnjx\") pod \"c871b3a0-4a8f-4113-9398-7af8c52b5a14\" (UID: \"c871b3a0-4a8f-4113-9398-7af8c52b5a14\") "
Sep 17 11:10:26.155431 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.155353    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c871b3a0-4a8f-4113-9398-7af8c52b5a14-plugins-config-volume\") pod \"c871b3a0-4a8f-4113-9398-7af8c52b5a14\" (UID: \"c871b3a0-4a8f-4113-9398-7af8c52b5a14\") "
Sep 17 11:10:26.155738 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.155703    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea7c5878-2b13-400c-bd90-c2ae2b11962c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ea7c5878-2b13-400c-bd90-c2ae2b11962c" (UID: "ea7c5878-2b13-400c-bd90-c2ae2b11962c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:10:26.155912 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.155845    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c871b3a0-4a8f-4113-9398-7af8c52b5a14-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c871b3a0-4a8f-4113-9398-7af8c52b5a14" (UID: "c871b3a0-4a8f-4113-9398-7af8c52b5a14"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:10:26.157678 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.157642    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea7c5878-2b13-400c-bd90-c2ae2b11962c-kube-api-access-tgmbq" (OuterVolumeSpecName: "kube-api-access-tgmbq") pod "ea7c5878-2b13-400c-bd90-c2ae2b11962c" (UID: "ea7c5878-2b13-400c-bd90-c2ae2b11962c"). InnerVolumeSpecName "kube-api-access-tgmbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:10:26.157770 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.157725    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c871b3a0-4a8f-4113-9398-7af8c52b5a14-kube-api-access-5pnjx" (OuterVolumeSpecName: "kube-api-access-5pnjx") pod "c871b3a0-4a8f-4113-9398-7af8c52b5a14" (UID: "c871b3a0-4a8f-4113-9398-7af8c52b5a14"). InnerVolumeSpecName "kube-api-access-5pnjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:10:26.188826 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.188798    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"]
Sep 17 11:10:26.189044 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.189016    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq" podUID="8380c71e-7700-4059-9266-8926a325650e" containerName="istio-proxy" containerID="cri-o://657e0a84f744a8e1cc8eefaa364bde3eb63608560d3060c3bda237044520db24" gracePeriod=30
Sep 17 11:10:26.257054 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.256992    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pnjx\" (UniqueName: \"kubernetes.io/projected/c871b3a0-4a8f-4113-9398-7af8c52b5a14-kube-api-access-5pnjx\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:10:26.257054 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.257022    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c871b3a0-4a8f-4113-9398-7af8c52b5a14-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:10:26.257054 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.257041    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgmbq\" (UniqueName: \"kubernetes.io/projected/ea7c5878-2b13-400c-bd90-c2ae2b11962c-kube-api-access-tgmbq\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:10:26.257054 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.257056    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea7c5878-2b13-400c-bd90-c2ae2b11962c-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:10:26.406296 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.406258    2564 generic.go:356] "Generic (PLEG): container finished" podID="ea7c5878-2b13-400c-bd90-c2ae2b11962c" containerID="d4326d46c221d98ed44adefc275c64729d3ef671a638164b23a4b498c5fba621" exitCode=0
Sep 17 11:10:26.406442 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.406312    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"
Sep 17 11:10:26.406442 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.406347    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4" event={"ID":"ea7c5878-2b13-400c-bd90-c2ae2b11962c","Type":"ContainerDied","Data":"d4326d46c221d98ed44adefc275c64729d3ef671a638164b23a4b498c5fba621"}
Sep 17 11:10:26.406442 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.406407    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4" event={"ID":"ea7c5878-2b13-400c-bd90-c2ae2b11962c","Type":"ContainerDied","Data":"05f9b7771a6b294fba59ac92a1d039782c98748f4b88db560cba5a1892207363"}
Sep 17 11:10:26.406442 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.406434    2564 scope.go:117] "RemoveContainer" containerID="d4326d46c221d98ed44adefc275c64729d3ef671a638164b23a4b498c5fba621"
Sep 17 11:10:26.407802 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.407785    2564 generic.go:356] "Generic (PLEG): container finished" podID="c871b3a0-4a8f-4113-9398-7af8c52b5a14" containerID="11086a23384c23573d0c73b8e27dbf90ac47cd6dfde7c0d544c75abdf0a88429" exitCode=0
Sep 17 11:10:26.407936 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.407832    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"
Sep 17 11:10:26.407936 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.407861    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6" event={"ID":"c871b3a0-4a8f-4113-9398-7af8c52b5a14","Type":"ContainerDied","Data":"11086a23384c23573d0c73b8e27dbf90ac47cd6dfde7c0d544c75abdf0a88429"}
Sep 17 11:10:26.407936 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.407928    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6" event={"ID":"c871b3a0-4a8f-4113-9398-7af8c52b5a14","Type":"ContainerDied","Data":"3bf263127782b8ac003951e354f6448e23082b47f9b23d54b58a4dae66dfacba"}
Sep 17 11:10:26.416518 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.416503    2564 scope.go:117] "RemoveContainer" containerID="d4326d46c221d98ed44adefc275c64729d3ef671a638164b23a4b498c5fba621"
Sep 17 11:10:26.416802 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:10:26.416776    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4326d46c221d98ed44adefc275c64729d3ef671a638164b23a4b498c5fba621\": container with ID starting with d4326d46c221d98ed44adefc275c64729d3ef671a638164b23a4b498c5fba621 not found: ID does not exist" containerID="d4326d46c221d98ed44adefc275c64729d3ef671a638164b23a4b498c5fba621"
Sep 17 11:10:26.417056 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.416813    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4326d46c221d98ed44adefc275c64729d3ef671a638164b23a4b498c5fba621"} err="failed to get container status \"d4326d46c221d98ed44adefc275c64729d3ef671a638164b23a4b498c5fba621\": rpc error: code = NotFound desc = could not find container \"d4326d46c221d98ed44adefc275c64729d3ef671a638164b23a4b498c5fba621\": container with ID starting with d4326d46c221d98ed44adefc275c64729d3ef671a638164b23a4b498c5fba621 not found: ID does not exist"
Sep 17 11:10:26.417056 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.416830    2564 scope.go:117] "RemoveContainer" containerID="11086a23384c23573d0c73b8e27dbf90ac47cd6dfde7c0d544c75abdf0a88429"
Sep 17 11:10:26.425230 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.425213    2564 scope.go:117] "RemoveContainer" containerID="11086a23384c23573d0c73b8e27dbf90ac47cd6dfde7c0d544c75abdf0a88429"
Sep 17 11:10:26.425475 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:10:26.425454    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11086a23384c23573d0c73b8e27dbf90ac47cd6dfde7c0d544c75abdf0a88429\": container with ID starting with 11086a23384c23573d0c73b8e27dbf90ac47cd6dfde7c0d544c75abdf0a88429 not found: ID does not exist" containerID="11086a23384c23573d0c73b8e27dbf90ac47cd6dfde7c0d544c75abdf0a88429"
Sep 17 11:10:26.425537 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.425483    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11086a23384c23573d0c73b8e27dbf90ac47cd6dfde7c0d544c75abdf0a88429"} err="failed to get container status \"11086a23384c23573d0c73b8e27dbf90ac47cd6dfde7c0d544c75abdf0a88429\": rpc error: code = NotFound desc = could not find container \"11086a23384c23573d0c73b8e27dbf90ac47cd6dfde7c0d544c75abdf0a88429\": container with ID starting with 11086a23384c23573d0c73b8e27dbf90ac47cd6dfde7c0d544c75abdf0a88429 not found: ID does not exist"
Sep 17 11:10:26.444806 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.444778    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"]
Sep 17 11:10:26.447393 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.447375    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-dfe3d913-57fb84ddc7-pgdl6"]
Sep 17 11:10:26.458724 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.458703    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"]
Sep 17 11:10:26.462192 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:26.462173    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-dfe3d913-5c99b5587d-wxpr4"]
Sep 17 11:10:27.146912 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.146875    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80e29cb4-f401-4f8f-99e4-50c481ad8acf" path="/var/lib/kubelet/pods/80e29cb4-f401-4f8f-99e4-50c481ad8acf/volumes"
Sep 17 11:10:27.147282 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.147266    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c871b3a0-4a8f-4113-9398-7af8c52b5a14" path="/var/lib/kubelet/pods/c871b3a0-4a8f-4113-9398-7af8c52b5a14/volumes"
Sep 17 11:10:27.147592 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.147580    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea7c5878-2b13-400c-bd90-c2ae2b11962c" path="/var/lib/kubelet/pods/ea7c5878-2b13-400c-bd90-c2ae2b11962c/volumes"
Sep 17 11:10:27.346247 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.346212    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"]
Sep 17 11:10:27.346633 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.346621    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80e29cb4-f401-4f8f-99e4-50c481ad8acf" containerName="revoke-keys"
Sep 17 11:10:27.346678 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.346635    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="80e29cb4-f401-4f8f-99e4-50c481ad8acf" containerName="revoke-keys"
Sep 17 11:10:27.346678 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.346652    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea7c5878-2b13-400c-bd90-c2ae2b11962c" containerName="payload-pre-processing"
Sep 17 11:10:27.346678 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.346658    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea7c5878-2b13-400c-bd90-c2ae2b11962c" containerName="payload-pre-processing"
Sep 17 11:10:27.346678 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.346670    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c871b3a0-4a8f-4113-9398-7af8c52b5a14" containerName="payload-processing"
Sep 17 11:10:27.346678 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.346677    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c871b3a0-4a8f-4113-9398-7af8c52b5a14" containerName="payload-processing"
Sep 17 11:10:27.346830 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.346741    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea7c5878-2b13-400c-bd90-c2ae2b11962c" containerName="payload-pre-processing"
Sep 17 11:10:27.346830 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.346747    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c871b3a0-4a8f-4113-9398-7af8c52b5a14" containerName="payload-processing"
Sep 17 11:10:27.346830 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.346755    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="80e29cb4-f401-4f8f-99e4-50c481ad8acf" containerName="revoke-keys"
Sep 17 11:10:27.354381 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.354354    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.357211 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.357179    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-76b12dde-gw-openshift-default-dockercfg-9j2gh\""
Sep 17 11:10:27.366956 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.366929    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-workload-socket\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.367088 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.366981    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-workload-certs\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.367157 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.367102    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-data\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.367157 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.367137    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-token\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.367270 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.367168    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22799997-0d86-4ba2-abd8-9f41125dc11f-istiod-ca-cert\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.367270 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.367213    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-credential-socket\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.367270 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.367251    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tbff\" (UniqueName: \"kubernetes.io/projected/22799997-0d86-4ba2-abd8-9f41125dc11f-kube-api-access-4tbff\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.367421 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.367314    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-envoy\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.367421 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.367364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-podinfo\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.372065 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.372040    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"]
Sep 17 11:10:27.468180 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.468095    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tbff\" (UniqueName: \"kubernetes.io/projected/22799997-0d86-4ba2-abd8-9f41125dc11f-kube-api-access-4tbff\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.468180 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.468143    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-envoy\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.468180 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.468178    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-podinfo\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.468464 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.468214    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-workload-socket\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.468464 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.468350    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-workload-certs\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.468579 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.468483    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-data\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.468579 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.468523    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-token\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.468579 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.468555    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22799997-0d86-4ba2-abd8-9f41125dc11f-istiod-ca-cert\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.468735 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.468597    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-credential-socket\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.468735 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.468675    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-workload-socket\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.468735 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.468725    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-workload-certs\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.468891 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.468805    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-data\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.469155 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.468961    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-credential-socket\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.469296 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.469277    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22799997-0d86-4ba2-abd8-9f41125dc11f-istiod-ca-cert\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.470703 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.470682    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-envoy\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.471036 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.471017    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-podinfo\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.477586 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.477558    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-token\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.478281 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.478265    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tbff\" (UniqueName: \"kubernetes.io/projected/22799997-0d86-4ba2-abd8-9f41125dc11f-kube-api-access-4tbff\") pod \"e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") " pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.665464 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.665429    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:27.946615 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.946585    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"]
Sep 17 11:10:27.947704 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:10:27.947675    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22799997_0d86_4ba2_abd8_9f41125dc11f.slice/crio-e64b566fae763e9ee573960f39eafcf082ae861103633a2636399862a3ef7c6f WatchSource:0}: Error finding container e64b566fae763e9ee573960f39eafcf082ae861103633a2636399862a3ef7c6f: Status 404 returned error can't find the container with id e64b566fae763e9ee573960f39eafcf082ae861103633a2636399862a3ef7c6f
Sep 17 11:10:27.950232 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.950197    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:10:27.950335 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.950285    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:10:27.950335 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:27.950329    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:10:28.423622 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:28.423537    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn" event={"ID":"22799997-0d86-4ba2-abd8-9f41125dc11f","Type":"ContainerStarted","Data":"faaa53466bd6bb1d71b4e519ecc86195133fcd013b6c74701ff36a36174b426f"}
Sep 17 11:10:28.423622 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:28.423588    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn" event={"ID":"22799997-0d86-4ba2-abd8-9f41125dc11f","Type":"ContainerStarted","Data":"e64b566fae763e9ee573960f39eafcf082ae861103633a2636399862a3ef7c6f"}
Sep 17 11:10:28.447490 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:28.447437    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn" podStartSLOduration=1.447422026 podStartE2EDuration="1.447422026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:10:28.444059746 +0000 UTC m=+1023.760422915" watchObservedRunningTime="2026-09-17 11:10:28.447422026 +0000 UTC m=+1023.763785181"
Sep 17 11:10:28.665854 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:28.665828    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:29.671053 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:29.671023    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:30.435247 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:30.435218    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:30.436111 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:30.436092    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:10:31.442448 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.442416    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:10:31.443627 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.443607    2564 generic.go:356] "Generic (PLEG): container finished" podID="8380c71e-7700-4059-9266-8926a325650e" containerID="657e0a84f744a8e1cc8eefaa364bde3eb63608560d3060c3bda237044520db24" exitCode=0
Sep 17 11:10:31.443726 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.443675    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq" event={"ID":"8380c71e-7700-4059-9266-8926a325650e","Type":"ContainerDied","Data":"657e0a84f744a8e1cc8eefaa364bde3eb63608560d3060c3bda237044520db24"}
Sep 17 11:10:31.443774 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.443727    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq" event={"ID":"8380c71e-7700-4059-9266-8926a325650e","Type":"ContainerDied","Data":"d54c377b39dbd38db48dbce85e633b90efac66587598d2485508a0ec1058137c"}
Sep 17 11:10:31.443774 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.443751    2564 scope.go:117] "RemoveContainer" containerID="657e0a84f744a8e1cc8eefaa364bde3eb63608560d3060c3bda237044520db24"
Sep 17 11:10:31.453250 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.453235    2564 scope.go:117] "RemoveContainer" containerID="75ede7068abd4044910db2d480c79476b5249cecbfbf4908822f74ea1e135d27"
Sep 17 11:10:31.462622 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.462598    2564 scope.go:117] "RemoveContainer" containerID="657e0a84f744a8e1cc8eefaa364bde3eb63608560d3060c3bda237044520db24"
Sep 17 11:10:31.462959 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:10:31.462936    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"657e0a84f744a8e1cc8eefaa364bde3eb63608560d3060c3bda237044520db24\": container with ID starting with 657e0a84f744a8e1cc8eefaa364bde3eb63608560d3060c3bda237044520db24 not found: ID does not exist" containerID="657e0a84f744a8e1cc8eefaa364bde3eb63608560d3060c3bda237044520db24"
Sep 17 11:10:31.463042 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.462965    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"657e0a84f744a8e1cc8eefaa364bde3eb63608560d3060c3bda237044520db24"} err="failed to get container status \"657e0a84f744a8e1cc8eefaa364bde3eb63608560d3060c3bda237044520db24\": rpc error: code = NotFound desc = could not find container \"657e0a84f744a8e1cc8eefaa364bde3eb63608560d3060c3bda237044520db24\": container with ID starting with 657e0a84f744a8e1cc8eefaa364bde3eb63608560d3060c3bda237044520db24 not found: ID does not exist"
Sep 17 11:10:31.463042 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.462982    2564 scope.go:117] "RemoveContainer" containerID="75ede7068abd4044910db2d480c79476b5249cecbfbf4908822f74ea1e135d27"
Sep 17 11:10:31.463268 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:10:31.463251    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75ede7068abd4044910db2d480c79476b5249cecbfbf4908822f74ea1e135d27\": container with ID starting with 75ede7068abd4044910db2d480c79476b5249cecbfbf4908822f74ea1e135d27 not found: ID does not exist" containerID="75ede7068abd4044910db2d480c79476b5249cecbfbf4908822f74ea1e135d27"
Sep 17 11:10:31.463318 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.463273    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75ede7068abd4044910db2d480c79476b5249cecbfbf4908822f74ea1e135d27"} err="failed to get container status \"75ede7068abd4044910db2d480c79476b5249cecbfbf4908822f74ea1e135d27\": rpc error: code = NotFound desc = could not find container \"75ede7068abd4044910db2d480c79476b5249cecbfbf4908822f74ea1e135d27\": container with ID starting with 75ede7068abd4044910db2d480c79476b5249cecbfbf4908822f74ea1e135d27 not found: ID does not exist"
Sep 17 11:10:31.510205 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.510176    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-workload-socket\") pod \"8380c71e-7700-4059-9266-8926a325650e\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") "
Sep 17 11:10:31.510374 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.510218    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-istio-envoy\") pod \"8380c71e-7700-4059-9266-8926a325650e\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") "
Sep 17 11:10:31.510374 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.510250    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-workload-certs\") pod \"8380c71e-7700-4059-9266-8926a325650e\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") "
Sep 17 11:10:31.510374 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.510280    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8380c71e-7700-4059-9266-8926a325650e-istiod-ca-cert\") pod \"8380c71e-7700-4059-9266-8926a325650e\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") "
Sep 17 11:10:31.510374 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.510306    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8380c71e-7700-4059-9266-8926a325650e-istio-podinfo\") pod \"8380c71e-7700-4059-9266-8926a325650e\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") "
Sep 17 11:10:31.510374 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.510348    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwjdj\" (UniqueName: \"kubernetes.io/projected/8380c71e-7700-4059-9266-8926a325650e-kube-api-access-lwjdj\") pod \"8380c71e-7700-4059-9266-8926a325650e\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") "
Sep 17 11:10:31.510658 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.510431    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-credential-socket\") pod \"8380c71e-7700-4059-9266-8926a325650e\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") "
Sep 17 11:10:31.510658 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.510460    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8380c71e-7700-4059-9266-8926a325650e-istio-token\") pod \"8380c71e-7700-4059-9266-8926a325650e\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") "
Sep 17 11:10:31.510658 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.510535    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-istio-data\") pod \"8380c71e-7700-4059-9266-8926a325650e\" (UID: \"8380c71e-7700-4059-9266-8926a325650e\") "
Sep 17 11:10:31.510658 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.510455    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8380c71e-7700-4059-9266-8926a325650e" (UID: "8380c71e-7700-4059-9266-8926a325650e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:10:31.510874 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.510747    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8380c71e-7700-4059-9266-8926a325650e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8380c71e-7700-4059-9266-8926a325650e" (UID: "8380c71e-7700-4059-9266-8926a325650e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:10:31.510874 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.510786    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8380c71e-7700-4059-9266-8926a325650e" (UID: "8380c71e-7700-4059-9266-8926a325650e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:10:31.511029 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.510960    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8380c71e-7700-4059-9266-8926a325650e" (UID: "8380c71e-7700-4059-9266-8926a325650e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:10:31.511088 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.511058    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-workload-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:10:31.511088 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.511078    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-workload-certs\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:10:31.511198 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.511092    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8380c71e-7700-4059-9266-8926a325650e-istiod-ca-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:10:31.511198 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.511105    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-credential-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:10:31.511198 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.511163    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-istio-data" (OuterVolumeSpecName: "istio-data") pod "8380c71e-7700-4059-9266-8926a325650e" (UID: "8380c71e-7700-4059-9266-8926a325650e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:10:31.513097 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.513071    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8380c71e-7700-4059-9266-8926a325650e" (UID: "8380c71e-7700-4059-9266-8926a325650e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:10:31.513224 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.513202    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8380c71e-7700-4059-9266-8926a325650e-istio-token" (OuterVolumeSpecName: "istio-token") pod "8380c71e-7700-4059-9266-8926a325650e" (UID: "8380c71e-7700-4059-9266-8926a325650e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:10:31.513473 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.513450    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8380c71e-7700-4059-9266-8926a325650e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8380c71e-7700-4059-9266-8926a325650e" (UID: "8380c71e-7700-4059-9266-8926a325650e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 11:10:31.513473 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.513461    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8380c71e-7700-4059-9266-8926a325650e-kube-api-access-lwjdj" (OuterVolumeSpecName: "kube-api-access-lwjdj") pod "8380c71e-7700-4059-9266-8926a325650e" (UID: "8380c71e-7700-4059-9266-8926a325650e"). InnerVolumeSpecName "kube-api-access-lwjdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:10:31.611952 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.611892    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-istio-data\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:10:31.611952 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.611943    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8380c71e-7700-4059-9266-8926a325650e-istio-envoy\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:10:31.611952 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.611952    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8380c71e-7700-4059-9266-8926a325650e-istio-podinfo\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:10:31.611952 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.611960    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwjdj\" (UniqueName: \"kubernetes.io/projected/8380c71e-7700-4059-9266-8926a325650e-kube-api-access-lwjdj\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:10:31.612209 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.611971    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8380c71e-7700-4059-9266-8926a325650e-istio-token\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:10:31.804589 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.804551    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"]
Sep 17 11:10:31.805088 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.805069    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8380c71e-7700-4059-9266-8926a325650e" containerName="istio-proxy"
Sep 17 11:10:31.805088 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.805089    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8380c71e-7700-4059-9266-8926a325650e" containerName="istio-proxy"
Sep 17 11:10:31.805267 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.805232    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8380c71e-7700-4059-9266-8926a325650e" containerName="istio-proxy"
Sep 17 11:10:31.805267 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.805246    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8380c71e-7700-4059-9266-8926a325650e" containerName="istio-proxy"
Sep 17 11:10:31.810963 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.810942    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"
Sep 17 11:10:31.812963 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.812931    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-76b12dde\""
Sep 17 11:10:31.813083 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.812970    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-76b12dde-dockercfg-9zl56\""
Sep 17 11:10:31.820680 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.820150    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"]
Sep 17 11:10:31.854345 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.854315    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"]
Sep 17 11:10:31.854816 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.854796    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8380c71e-7700-4059-9266-8926a325650e" containerName="istio-proxy"
Sep 17 11:10:31.854816 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.854815    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8380c71e-7700-4059-9266-8926a325650e" containerName="istio-proxy"
Sep 17 11:10:31.858125 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.858109    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"
Sep 17 11:10:31.868813 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.868790    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"]
Sep 17 11:10:31.915923 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.915222    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmvlb\" (UniqueName: \"kubernetes.io/projected/70f99b11-cb71-4999-9b71-2c43ecaf9427-kube-api-access-zmvlb\") pod \"payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2\" (UID: \"70f99b11-cb71-4999-9b71-2c43ecaf9427\") " pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"
Sep 17 11:10:31.915923 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.915360    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15639e0d-552e-41c3-bbb6-6e3e2f59d7df-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph\" (UID: \"15639e0d-552e-41c3-bbb6-6e3e2f59d7df\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"
Sep 17 11:10:31.915923 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.915390    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70f99b11-cb71-4999-9b71-2c43ecaf9427-plugins-config-volume\") pod \"payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2\" (UID: \"70f99b11-cb71-4999-9b71-2c43ecaf9427\") " pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"
Sep 17 11:10:31.915923 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:31.915449    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mn4v\" (UniqueName: \"kubernetes.io/projected/15639e0d-552e-41c3-bbb6-6e3e2f59d7df-kube-api-access-4mn4v\") pod \"payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph\" (UID: \"15639e0d-552e-41c3-bbb6-6e3e2f59d7df\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"
Sep 17 11:10:32.016196 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:32.016162    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15639e0d-552e-41c3-bbb6-6e3e2f59d7df-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph\" (UID: \"15639e0d-552e-41c3-bbb6-6e3e2f59d7df\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"
Sep 17 11:10:32.016196 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:32.016195    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70f99b11-cb71-4999-9b71-2c43ecaf9427-plugins-config-volume\") pod \"payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2\" (UID: \"70f99b11-cb71-4999-9b71-2c43ecaf9427\") " pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"
Sep 17 11:10:32.016449 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:32.016231    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mn4v\" (UniqueName: \"kubernetes.io/projected/15639e0d-552e-41c3-bbb6-6e3e2f59d7df-kube-api-access-4mn4v\") pod \"payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph\" (UID: \"15639e0d-552e-41c3-bbb6-6e3e2f59d7df\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"
Sep 17 11:10:32.016449 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:32.016266    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmvlb\" (UniqueName: \"kubernetes.io/projected/70f99b11-cb71-4999-9b71-2c43ecaf9427-kube-api-access-zmvlb\") pod \"payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2\" (UID: \"70f99b11-cb71-4999-9b71-2c43ecaf9427\") " pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"
Sep 17 11:10:32.016975 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:32.016945    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70f99b11-cb71-4999-9b71-2c43ecaf9427-plugins-config-volume\") pod \"payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2\" (UID: \"70f99b11-cb71-4999-9b71-2c43ecaf9427\") " pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"
Sep 17 11:10:32.017079 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:32.016953    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15639e0d-552e-41c3-bbb6-6e3e2f59d7df-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph\" (UID: \"15639e0d-552e-41c3-bbb6-6e3e2f59d7df\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"
Sep 17 11:10:32.024178 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:32.024159    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mn4v\" (UniqueName: \"kubernetes.io/projected/15639e0d-552e-41c3-bbb6-6e3e2f59d7df-kube-api-access-4mn4v\") pod \"payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph\" (UID: \"15639e0d-552e-41c3-bbb6-6e3e2f59d7df\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"
Sep 17 11:10:32.024314 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:32.024297    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmvlb\" (UniqueName: \"kubernetes.io/projected/70f99b11-cb71-4999-9b71-2c43ecaf9427-kube-api-access-zmvlb\") pod \"payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2\" (UID: \"70f99b11-cb71-4999-9b71-2c43ecaf9427\") " pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"
Sep 17 11:10:32.124408 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:32.124331    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"
Sep 17 11:10:32.169736 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:32.169710    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"
Sep 17 11:10:32.399779 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:32.399739    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"]
Sep 17 11:10:32.400304 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:10:32.400283    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70f99b11_cb71_4999_9b71_2c43ecaf9427.slice/crio-01f2e23ef13ba89a2d33d2b6ca3da39b81755016471c0e31088f648c9cdfe364 WatchSource:0}: Error finding container 01f2e23ef13ba89a2d33d2b6ca3da39b81755016471c0e31088f648c9cdfe364: Status 404 returned error can't find the container with id 01f2e23ef13ba89a2d33d2b6ca3da39b81755016471c0e31088f648c9cdfe364
Sep 17 11:10:32.450520 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:32.450492    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2" event={"ID":"70f99b11-cb71-4999-9b71-2c43ecaf9427","Type":"ContainerStarted","Data":"01f2e23ef13ba89a2d33d2b6ca3da39b81755016471c0e31088f648c9cdfe364"}
Sep 17 11:10:32.451594 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:32.451573    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"
Sep 17 11:10:32.460781 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:32.460765    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"]
Sep 17 11:10:32.461578 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:10:32.461556    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15639e0d_552e_41c3_bbb6_6e3e2f59d7df.slice/crio-6e662b7def6139336321647c23ec0bf04f32599c3c7bb5b96debb1a78e3434e4 WatchSource:0}: Error finding container 6e662b7def6139336321647c23ec0bf04f32599c3c7bb5b96debb1a78e3434e4: Status 404 returned error can't find the container with id 6e662b7def6139336321647c23ec0bf04f32599c3c7bb5b96debb1a78e3434e4
Sep 17 11:10:32.487326 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:32.487285    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"]
Sep 17 11:10:32.493595 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:32.493572    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-dfe3d913-openshift-default-9c74c78cd-fpzgq"]
Sep 17 11:10:33.145055 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:33.145016    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8380c71e-7700-4059-9266-8926a325650e" path="/var/lib/kubelet/pods/8380c71e-7700-4059-9266-8926a325650e/volumes"
Sep 17 11:10:33.459237 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:33.459147    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph" event={"ID":"15639e0d-552e-41c3-bbb6-6e3e2f59d7df","Type":"ContainerStarted","Data":"f6f27a557eb77d50678b335f97a18f86dbb5d7984825fb5e5816e2dd7b71134b"}
Sep 17 11:10:33.459237 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:33.459213    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"
Sep 17 11:10:33.459686 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:33.459239    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph" event={"ID":"15639e0d-552e-41c3-bbb6-6e3e2f59d7df","Type":"ContainerStarted","Data":"6e662b7def6139336321647c23ec0bf04f32599c3c7bb5b96debb1a78e3434e4"}
Sep 17 11:10:33.460779 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:33.460754    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2" event={"ID":"70f99b11-cb71-4999-9b71-2c43ecaf9427","Type":"ContainerStarted","Data":"50c249691fafcc0890d9c5028d7f6356b862e064c4453df988e8b51e5421e714"}
Sep 17 11:10:33.460945 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:33.460887    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"
Sep 17 11:10:33.484098 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:33.484054    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph" podStartSLOduration=2.484037092 podStartE2EDuration="2.484037092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:10:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:10:33.479976175 +0000 UTC m=+1028.796339328" watchObservedRunningTime="2026-09-17 11:10:33.484037092 +0000 UTC m=+1028.800400246"
Sep 17 11:10:33.505315 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:33.505275    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2" podStartSLOduration=2.5052616370000003 podStartE2EDuration="2.505261637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:10:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:10:33.502918337 +0000 UTC m=+1028.819281481" watchObservedRunningTime="2026-09-17 11:10:33.505261637 +0000 UTC m=+1028.821624792"
Sep 17 11:10:44.468085 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:44.468057    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"
Sep 17 11:10:44.468507 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:44.468439    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"
Sep 17 11:10:54.923169 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:54.923132    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8"]
Sep 17 11:10:54.925692 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:54.925675    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8"
Sep 17 11:10:54.927951 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:54.927925    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:10:54.928090 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:54.928012    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:10:54.939041 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:54.939016    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8"]
Sep 17 11:10:55.040991 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:55.040946    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebfd167d-189d-40c7-86c9-9a151cc285e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8\" (UID: \"ebfd167d-189d-40c7-86c9-9a151cc285e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8"
Sep 17 11:10:55.141705 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:55.141665    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebfd167d-189d-40c7-86c9-9a151cc285e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8\" (UID: \"ebfd167d-189d-40c7-86c9-9a151cc285e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8"
Sep 17 11:10:55.142289 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:55.142264    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebfd167d-189d-40c7-86c9-9a151cc285e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8\" (UID: \"ebfd167d-189d-40c7-86c9-9a151cc285e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8"
Sep 17 11:10:55.238419 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:55.238389    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8"
Sep 17 11:10:55.515758 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:55.515732    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8"]
Sep 17 11:10:55.517589 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:10:55.517560    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebfd167d_189d_40c7_86c9_9a151cc285e4.slice/crio-c4da91af8cdf5fbfcd70d7160cb179c9f39036e8bcdb9b5f67835418c8521e60 WatchSource:0}: Error finding container c4da91af8cdf5fbfcd70d7160cb179c9f39036e8bcdb9b5f67835418c8521e60: Status 404 returned error can't find the container with id c4da91af8cdf5fbfcd70d7160cb179c9f39036e8bcdb9b5f67835418c8521e60
Sep 17 11:10:55.590945 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:55.590909    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8" event={"ID":"ebfd167d-189d-40c7-86c9-9a151cc285e4","Type":"ContainerStarted","Data":"c4da91af8cdf5fbfcd70d7160cb179c9f39036e8bcdb9b5f67835418c8521e60"}
Sep 17 11:10:56.601610 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:56.601524    2564 generic.go:356] "Generic (PLEG): container finished" podID="ebfd167d-189d-40c7-86c9-9a151cc285e4" containerID="f22c20aa92fe979e422ba6caad7fadde2af04d2fd80df95a6d9b7c4289cdc4db" exitCode=0
Sep 17 11:10:56.601992 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:56.601614    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8" event={"ID":"ebfd167d-189d-40c7-86c9-9a151cc285e4","Type":"ContainerDied","Data":"f22c20aa92fe979e422ba6caad7fadde2af04d2fd80df95a6d9b7c4289cdc4db"}
Sep 17 11:10:57.784881 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:57.784845    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8"
Sep 17 11:10:57.868414 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:57.868330    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebfd167d-189d-40c7-86c9-9a151cc285e4-maas-api-service-ca\") pod \"ebfd167d-189d-40c7-86c9-9a151cc285e4\" (UID: \"ebfd167d-189d-40c7-86c9-9a151cc285e4\") "
Sep 17 11:10:57.868652 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:57.868630    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebfd167d-189d-40c7-86c9-9a151cc285e4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ebfd167d-189d-40c7-86c9-9a151cc285e4" (UID: "ebfd167d-189d-40c7-86c9-9a151cc285e4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:10:57.868723 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:57.868711    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebfd167d-189d-40c7-86c9-9a151cc285e4-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:10:58.619368 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:58.619336    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8"
Sep 17 11:10:58.619538 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:58.619374    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8" event={"ID":"ebfd167d-189d-40c7-86c9-9a151cc285e4","Type":"ContainerDied","Data":"c4da91af8cdf5fbfcd70d7160cb179c9f39036e8bcdb9b5f67835418c8521e60"}
Sep 17 11:10:58.619538 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:10:58.619422    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4da91af8cdf5fbfcd70d7160cb179c9f39036e8bcdb9b5f67835418c8521e60"
Sep 17 11:11:01.962409 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:01.962374    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"]
Sep 17 11:11:01.962791 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:01.962590    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2" podUID="70f99b11-cb71-4999-9b71-2c43ecaf9427" containerName="payload-processing" containerID="cri-o://50c249691fafcc0890d9c5028d7f6356b862e064c4453df988e8b51e5421e714" gracePeriod=30
Sep 17 11:11:01.975085 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:01.975059    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"]
Sep 17 11:11:01.975495 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:01.975467    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph" podUID="15639e0d-552e-41c3-bbb6-6e3e2f59d7df" containerName="payload-pre-processing" containerID="cri-o://f6f27a557eb77d50678b335f97a18f86dbb5d7984825fb5e5816e2dd7b71134b" gracePeriod=30
Sep 17 11:11:02.183589 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.181878    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8"]
Sep 17 11:11:02.208923 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.199466    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-76b12dde-whkf8"]
Sep 17 11:11:02.330951 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.322149    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"
Sep 17 11:11:02.334853 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.334618    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"]
Sep 17 11:11:02.335333 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.335291    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn" podUID="22799997-0d86-4ba2-abd8-9f41125dc11f" containerName="istio-proxy" containerID="cri-o://faaa53466bd6bb1d71b4e519ecc86195133fcd013b6c74701ff36a36174b426f" gracePeriod=30
Sep 17 11:11:02.362783 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.362764    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"
Sep 17 11:11:02.424332 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.424285    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmvlb\" (UniqueName: \"kubernetes.io/projected/70f99b11-cb71-4999-9b71-2c43ecaf9427-kube-api-access-zmvlb\") pod \"70f99b11-cb71-4999-9b71-2c43ecaf9427\" (UID: \"70f99b11-cb71-4999-9b71-2c43ecaf9427\") "
Sep 17 11:11:02.424738 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.424714    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70f99b11-cb71-4999-9b71-2c43ecaf9427-plugins-config-volume\") pod \"70f99b11-cb71-4999-9b71-2c43ecaf9427\" (UID: \"70f99b11-cb71-4999-9b71-2c43ecaf9427\") "
Sep 17 11:11:02.425727 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.425702    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70f99b11-cb71-4999-9b71-2c43ecaf9427-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "70f99b11-cb71-4999-9b71-2c43ecaf9427" (UID: "70f99b11-cb71-4999-9b71-2c43ecaf9427"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:11:02.428641 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.428606    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70f99b11-cb71-4999-9b71-2c43ecaf9427-kube-api-access-zmvlb" (OuterVolumeSpecName: "kube-api-access-zmvlb") pod "70f99b11-cb71-4999-9b71-2c43ecaf9427" (UID: "70f99b11-cb71-4999-9b71-2c43ecaf9427"). InnerVolumeSpecName "kube-api-access-zmvlb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:11:02.525752 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.525655    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15639e0d-552e-41c3-bbb6-6e3e2f59d7df-plugins-config-volume\") pod \"15639e0d-552e-41c3-bbb6-6e3e2f59d7df\" (UID: \"15639e0d-552e-41c3-bbb6-6e3e2f59d7df\") "
Sep 17 11:11:02.525752 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.525703    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mn4v\" (UniqueName: \"kubernetes.io/projected/15639e0d-552e-41c3-bbb6-6e3e2f59d7df-kube-api-access-4mn4v\") pod \"15639e0d-552e-41c3-bbb6-6e3e2f59d7df\" (UID: \"15639e0d-552e-41c3-bbb6-6e3e2f59d7df\") "
Sep 17 11:11:02.525993 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.525947    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70f99b11-cb71-4999-9b71-2c43ecaf9427-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:02.525993 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.525960    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmvlb\" (UniqueName: \"kubernetes.io/projected/70f99b11-cb71-4999-9b71-2c43ecaf9427-kube-api-access-zmvlb\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:02.526088 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.526061    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15639e0d-552e-41c3-bbb6-6e3e2f59d7df-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "15639e0d-552e-41c3-bbb6-6e3e2f59d7df" (UID: "15639e0d-552e-41c3-bbb6-6e3e2f59d7df"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:11:02.527960 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.527933    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15639e0d-552e-41c3-bbb6-6e3e2f59d7df-kube-api-access-4mn4v" (OuterVolumeSpecName: "kube-api-access-4mn4v") pod "15639e0d-552e-41c3-bbb6-6e3e2f59d7df" (UID: "15639e0d-552e-41c3-bbb6-6e3e2f59d7df"). InnerVolumeSpecName "kube-api-access-4mn4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:11:02.626727 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.626685    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15639e0d-552e-41c3-bbb6-6e3e2f59d7df-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:02.626727 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.626717    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mn4v\" (UniqueName: \"kubernetes.io/projected/15639e0d-552e-41c3-bbb6-6e3e2f59d7df-kube-api-access-4mn4v\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:02.645352 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.645321    2564 generic.go:356] "Generic (PLEG): container finished" podID="70f99b11-cb71-4999-9b71-2c43ecaf9427" containerID="50c249691fafcc0890d9c5028d7f6356b862e064c4453df988e8b51e5421e714" exitCode=0
Sep 17 11:11:02.645499 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.645370    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"
Sep 17 11:11:02.645499 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.645408    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2" event={"ID":"70f99b11-cb71-4999-9b71-2c43ecaf9427","Type":"ContainerDied","Data":"50c249691fafcc0890d9c5028d7f6356b862e064c4453df988e8b51e5421e714"}
Sep 17 11:11:02.645499 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.645461    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2" event={"ID":"70f99b11-cb71-4999-9b71-2c43ecaf9427","Type":"ContainerDied","Data":"01f2e23ef13ba89a2d33d2b6ca3da39b81755016471c0e31088f648c9cdfe364"}
Sep 17 11:11:02.645499 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.645490    2564 scope.go:117] "RemoveContainer" containerID="50c249691fafcc0890d9c5028d7f6356b862e064c4453df988e8b51e5421e714"
Sep 17 11:11:02.646873 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.646847    2564 generic.go:356] "Generic (PLEG): container finished" podID="15639e0d-552e-41c3-bbb6-6e3e2f59d7df" containerID="f6f27a557eb77d50678b335f97a18f86dbb5d7984825fb5e5816e2dd7b71134b" exitCode=0
Sep 17 11:11:02.647017 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.646890    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"
Sep 17 11:11:02.647017 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.646923    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph" event={"ID":"15639e0d-552e-41c3-bbb6-6e3e2f59d7df","Type":"ContainerDied","Data":"f6f27a557eb77d50678b335f97a18f86dbb5d7984825fb5e5816e2dd7b71134b"}
Sep 17 11:11:02.647017 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.646963    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph" event={"ID":"15639e0d-552e-41c3-bbb6-6e3e2f59d7df","Type":"ContainerDied","Data":"6e662b7def6139336321647c23ec0bf04f32599c3c7bb5b96debb1a78e3434e4"}
Sep 17 11:11:02.657283 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.657265    2564 scope.go:117] "RemoveContainer" containerID="50c249691fafcc0890d9c5028d7f6356b862e064c4453df988e8b51e5421e714"
Sep 17 11:11:02.657661 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:11:02.657641    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50c249691fafcc0890d9c5028d7f6356b862e064c4453df988e8b51e5421e714\": container with ID starting with 50c249691fafcc0890d9c5028d7f6356b862e064c4453df988e8b51e5421e714 not found: ID does not exist" containerID="50c249691fafcc0890d9c5028d7f6356b862e064c4453df988e8b51e5421e714"
Sep 17 11:11:02.657737 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.657669    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50c249691fafcc0890d9c5028d7f6356b862e064c4453df988e8b51e5421e714"} err="failed to get container status \"50c249691fafcc0890d9c5028d7f6356b862e064c4453df988e8b51e5421e714\": rpc error: code = NotFound desc = could not find container \"50c249691fafcc0890d9c5028d7f6356b862e064c4453df988e8b51e5421e714\": container with ID starting with 50c249691fafcc0890d9c5028d7f6356b862e064c4453df988e8b51e5421e714 not found: ID does not exist"
Sep 17 11:11:02.657737 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.657684    2564 scope.go:117] "RemoveContainer" containerID="f6f27a557eb77d50678b335f97a18f86dbb5d7984825fb5e5816e2dd7b71134b"
Sep 17 11:11:02.666054 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.666035    2564 scope.go:117] "RemoveContainer" containerID="f6f27a557eb77d50678b335f97a18f86dbb5d7984825fb5e5816e2dd7b71134b"
Sep 17 11:11:02.666321 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:11:02.666304    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6f27a557eb77d50678b335f97a18f86dbb5d7984825fb5e5816e2dd7b71134b\": container with ID starting with f6f27a557eb77d50678b335f97a18f86dbb5d7984825fb5e5816e2dd7b71134b not found: ID does not exist" containerID="f6f27a557eb77d50678b335f97a18f86dbb5d7984825fb5e5816e2dd7b71134b"
Sep 17 11:11:02.666378 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.666327    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6f27a557eb77d50678b335f97a18f86dbb5d7984825fb5e5816e2dd7b71134b"} err="failed to get container status \"f6f27a557eb77d50678b335f97a18f86dbb5d7984825fb5e5816e2dd7b71134b\": rpc error: code = NotFound desc = could not find container \"f6f27a557eb77d50678b335f97a18f86dbb5d7984825fb5e5816e2dd7b71134b\": container with ID starting with f6f27a557eb77d50678b335f97a18f86dbb5d7984825fb5e5816e2dd7b71134b not found: ID does not exist"
Sep 17 11:11:02.680384 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.680356    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"]
Sep 17 11:11:02.685274 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.685252    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-76b12dde-75997ffb5d-zrvph"]
Sep 17 11:11:02.709466 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.709444    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"]
Sep 17 11:11:02.714038 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:02.714017    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-76b12dde-5f6576689d-w5xf2"]
Sep 17 11:11:03.131691 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.131655    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"]
Sep 17 11:11:03.132310 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.132290    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebfd167d-189d-40c7-86c9-9a151cc285e4" containerName="revoke-keys"
Sep 17 11:11:03.132401 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.132312    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebfd167d-189d-40c7-86c9-9a151cc285e4" containerName="revoke-keys"
Sep 17 11:11:03.132401 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.132328    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70f99b11-cb71-4999-9b71-2c43ecaf9427" containerName="payload-processing"
Sep 17 11:11:03.132401 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.132336    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="70f99b11-cb71-4999-9b71-2c43ecaf9427" containerName="payload-processing"
Sep 17 11:11:03.132401 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.132346    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15639e0d-552e-41c3-bbb6-6e3e2f59d7df" containerName="payload-pre-processing"
Sep 17 11:11:03.132401 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.132354    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="15639e0d-552e-41c3-bbb6-6e3e2f59d7df" containerName="payload-pre-processing"
Sep 17 11:11:03.132631 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.132473    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="15639e0d-552e-41c3-bbb6-6e3e2f59d7df" containerName="payload-pre-processing"
Sep 17 11:11:03.132631 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.132487    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="70f99b11-cb71-4999-9b71-2c43ecaf9427" containerName="payload-processing"
Sep 17 11:11:03.132631 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.132498    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebfd167d-189d-40c7-86c9-9a151cc285e4" containerName="revoke-keys"
Sep 17 11:11:03.135395 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.135376    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.139511 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.139492    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-15da354a-openshift-default-dockercfg-sd2j5\""
Sep 17 11:11:03.144491 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.144467    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15639e0d-552e-41c3-bbb6-6e3e2f59d7df" path="/var/lib/kubelet/pods/15639e0d-552e-41c3-bbb6-6e3e2f59d7df/volumes"
Sep 17 11:11:03.144915 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.144876    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70f99b11-cb71-4999-9b71-2c43ecaf9427" path="/var/lib/kubelet/pods/70f99b11-cb71-4999-9b71-2c43ecaf9427/volumes"
Sep 17 11:11:03.145253 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.145236    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebfd167d-189d-40c7-86c9-9a151cc285e4" path="/var/lib/kubelet/pods/ebfd167d-189d-40c7-86c9-9a151cc285e4/volumes"
Sep 17 11:11:03.157334 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.157312    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"]
Sep 17 11:11:03.234212 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.232731    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-token\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.234212 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.232798    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-envoy\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.234212 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.232832    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3742dddb-316d-4bfe-9bb6-5327545bafaa-istiod-ca-cert\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.234212 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.232926    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-podinfo\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.234212 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.232968    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-data\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.234212 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.233081    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-workload-certs\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.234212 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.233111    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-credential-socket\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.234212 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.233239    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-workload-socket\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.234212 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.233290    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfkh8\" (UniqueName: \"kubernetes.io/projected/3742dddb-316d-4bfe-9bb6-5327545bafaa-kube-api-access-dfkh8\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.334426 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.334393    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-envoy\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.334426 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.334426    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3742dddb-316d-4bfe-9bb6-5327545bafaa-istiod-ca-cert\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.334680 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.334453    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-podinfo\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.334680 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.334480    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-data\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.334680 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.334545    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-workload-certs\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.334680 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.334569    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-credential-socket\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.334680 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.334665    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-workload-socket\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.334889 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.334708    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfkh8\" (UniqueName: \"kubernetes.io/projected/3742dddb-316d-4bfe-9bb6-5327545bafaa-kube-api-access-dfkh8\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.334889 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.334737    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-token\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.335005 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.334971    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-data\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.335329 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.335228    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-workload-socket\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.335329 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.335288    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3742dddb-316d-4bfe-9bb6-5327545bafaa-istiod-ca-cert\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.335709 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.335483    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-workload-certs\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.335709 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.335687    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-credential-socket\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.336919 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.336880    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-envoy\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.337260 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.337244    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-podinfo\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.343001 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.342975    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-token\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.343118 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.343107    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfkh8\" (UniqueName: \"kubernetes.io/projected/3742dddb-316d-4bfe-9bb6-5327545bafaa-kube-api-access-dfkh8\") pod \"e2e-ait-15da354a-openshift-default-5f7499488-4jfc8\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") " pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.446688 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.446651    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:03.745919 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.745877    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"]
Sep 17 11:11:03.747437 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:11:03.747406    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3742dddb_316d_4bfe_9bb6_5327545bafaa.slice/crio-011cec1f6702b45a751d80ae6dd7f083336acd30ee76f2a2955897f021e6beb3 WatchSource:0}: Error finding container 011cec1f6702b45a751d80ae6dd7f083336acd30ee76f2a2955897f021e6beb3: Status 404 returned error can't find the container with id 011cec1f6702b45a751d80ae6dd7f083336acd30ee76f2a2955897f021e6beb3
Sep 17 11:11:03.757280 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.757236    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:11:03.757379 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.757327    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:11:03.757379 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:03.757366    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:11:04.664169 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:04.664131    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8" event={"ID":"3742dddb-316d-4bfe-9bb6-5327545bafaa","Type":"ContainerStarted","Data":"40d3554005f884ed2f13cfca6247a23cdea499c15ead918f3428584b3d7b328b"}
Sep 17 11:11:04.664628 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:04.664234    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8" event={"ID":"3742dddb-316d-4bfe-9bb6-5327545bafaa","Type":"ContainerStarted","Data":"011cec1f6702b45a751d80ae6dd7f083336acd30ee76f2a2955897f021e6beb3"}
Sep 17 11:11:04.690671 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:04.690609    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8" podStartSLOduration=1.690586971 podStartE2EDuration="1.690586971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:11:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:11:04.686053386 +0000 UTC m=+1060.002416541" watchObservedRunningTime="2026-09-17 11:11:04.690586971 +0000 UTC m=+1060.006950126"
Sep 17 11:11:05.447166 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:05.447123    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:05.452624 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:05.452601    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:05.670191 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:05.670158    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:05.671109 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:05.671087    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:06.100254 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.100232    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:11:06.137753 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.137679    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:11:06.265164 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.265134    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-credential-socket\") pod \"22799997-0d86-4ba2-abd8-9f41125dc11f\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") "
Sep 17 11:11:06.265345 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.265183    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-podinfo\") pod \"22799997-0d86-4ba2-abd8-9f41125dc11f\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") "
Sep 17 11:11:06.265345 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.265205    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-workload-certs\") pod \"22799997-0d86-4ba2-abd8-9f41125dc11f\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") "
Sep 17 11:11:06.265345 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.265240    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tbff\" (UniqueName: \"kubernetes.io/projected/22799997-0d86-4ba2-abd8-9f41125dc11f-kube-api-access-4tbff\") pod \"22799997-0d86-4ba2-abd8-9f41125dc11f\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") "
Sep 17 11:11:06.265345 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.265295    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-data\") pod \"22799997-0d86-4ba2-abd8-9f41125dc11f\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") "
Sep 17 11:11:06.265569 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.265372    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-workload-socket\") pod \"22799997-0d86-4ba2-abd8-9f41125dc11f\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") "
Sep 17 11:11:06.265569 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.265419    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22799997-0d86-4ba2-abd8-9f41125dc11f-istiod-ca-cert\") pod \"22799997-0d86-4ba2-abd8-9f41125dc11f\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") "
Sep 17 11:11:06.265569 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.265448    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "22799997-0d86-4ba2-abd8-9f41125dc11f" (UID: "22799997-0d86-4ba2-abd8-9f41125dc11f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:11:06.265569 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.265464    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "22799997-0d86-4ba2-abd8-9f41125dc11f" (UID: "22799997-0d86-4ba2-abd8-9f41125dc11f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:11:06.265569 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.265460    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-token\") pod \"22799997-0d86-4ba2-abd8-9f41125dc11f\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") "
Sep 17 11:11:06.265569 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.265532    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-envoy\") pod \"22799997-0d86-4ba2-abd8-9f41125dc11f\" (UID: \"22799997-0d86-4ba2-abd8-9f41125dc11f\") "
Sep 17 11:11:06.265849 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.265668    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "22799997-0d86-4ba2-abd8-9f41125dc11f" (UID: "22799997-0d86-4ba2-abd8-9f41125dc11f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:11:06.265849 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.265672    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-data" (OuterVolumeSpecName: "istio-data") pod "22799997-0d86-4ba2-abd8-9f41125dc11f" (UID: "22799997-0d86-4ba2-abd8-9f41125dc11f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:11:06.265849 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.265786    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22799997-0d86-4ba2-abd8-9f41125dc11f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "22799997-0d86-4ba2-abd8-9f41125dc11f" (UID: "22799997-0d86-4ba2-abd8-9f41125dc11f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:11:06.266312 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.266290    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-credential-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:06.266435 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.266315    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-workload-certs\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:06.266435 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.266330    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-data\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:06.266435 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.266345    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-workload-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:06.266435 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.266358    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22799997-0d86-4ba2-abd8-9f41125dc11f-istiod-ca-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:06.267721 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.267696    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "22799997-0d86-4ba2-abd8-9f41125dc11f" (UID: "22799997-0d86-4ba2-abd8-9f41125dc11f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 11:11:06.267721 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.267713    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22799997-0d86-4ba2-abd8-9f41125dc11f-kube-api-access-4tbff" (OuterVolumeSpecName: "kube-api-access-4tbff") pod "22799997-0d86-4ba2-abd8-9f41125dc11f" (UID: "22799997-0d86-4ba2-abd8-9f41125dc11f"). InnerVolumeSpecName "kube-api-access-4tbff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:11:06.268191 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.268168    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-token" (OuterVolumeSpecName: "istio-token") pod "22799997-0d86-4ba2-abd8-9f41125dc11f" (UID: "22799997-0d86-4ba2-abd8-9f41125dc11f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:11:06.268191 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.268180    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "22799997-0d86-4ba2-abd8-9f41125dc11f" (UID: "22799997-0d86-4ba2-abd8-9f41125dc11f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:11:06.367489 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.367457    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-token\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:06.367489 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.367484    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-envoy\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:06.367489 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.367493    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22799997-0d86-4ba2-abd8-9f41125dc11f-istio-podinfo\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:06.367717 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.367501    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tbff\" (UniqueName: \"kubernetes.io/projected/22799997-0d86-4ba2-abd8-9f41125dc11f-kube-api-access-4tbff\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:06.677783 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.677749    2564 generic.go:356] "Generic (PLEG): container finished" podID="22799997-0d86-4ba2-abd8-9f41125dc11f" containerID="faaa53466bd6bb1d71b4e519ecc86195133fcd013b6c74701ff36a36174b426f" exitCode=0
Sep 17 11:11:06.678278 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.677790    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"
Sep 17 11:11:06.678278 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.677833    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn" event={"ID":"22799997-0d86-4ba2-abd8-9f41125dc11f","Type":"ContainerDied","Data":"faaa53466bd6bb1d71b4e519ecc86195133fcd013b6c74701ff36a36174b426f"}
Sep 17 11:11:06.678278 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.677918    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn" event={"ID":"22799997-0d86-4ba2-abd8-9f41125dc11f","Type":"ContainerDied","Data":"e64b566fae763e9ee573960f39eafcf082ae861103633a2636399862a3ef7c6f"}
Sep 17 11:11:06.678278 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.677924    2564 scope.go:117] "RemoveContainer" containerID="faaa53466bd6bb1d71b4e519ecc86195133fcd013b6c74701ff36a36174b426f"
Sep 17 11:11:06.688480 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.688464    2564 scope.go:117] "RemoveContainer" containerID="faaa53466bd6bb1d71b4e519ecc86195133fcd013b6c74701ff36a36174b426f"
Sep 17 11:11:06.688734 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:11:06.688715    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faaa53466bd6bb1d71b4e519ecc86195133fcd013b6c74701ff36a36174b426f\": container with ID starting with faaa53466bd6bb1d71b4e519ecc86195133fcd013b6c74701ff36a36174b426f not found: ID does not exist" containerID="faaa53466bd6bb1d71b4e519ecc86195133fcd013b6c74701ff36a36174b426f"
Sep 17 11:11:06.688782 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.688743    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faaa53466bd6bb1d71b4e519ecc86195133fcd013b6c74701ff36a36174b426f"} err="failed to get container status \"faaa53466bd6bb1d71b4e519ecc86195133fcd013b6c74701ff36a36174b426f\": rpc error: code = NotFound desc = could not find container \"faaa53466bd6bb1d71b4e519ecc86195133fcd013b6c74701ff36a36174b426f\": container with ID starting with faaa53466bd6bb1d71b4e519ecc86195133fcd013b6c74701ff36a36174b426f not found: ID does not exist"
Sep 17 11:11:06.702838 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.702782    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"]
Sep 17 11:11:06.705642 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:06.705623    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-76b12dde-gw-openshift-default-7b4949c988-p85dn"]
Sep 17 11:11:07.021616 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:07.021579    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7"]
Sep 17 11:11:07.022022 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:07.022008    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22799997-0d86-4ba2-abd8-9f41125dc11f" containerName="istio-proxy"
Sep 17 11:11:07.022075 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:07.022023    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="22799997-0d86-4ba2-abd8-9f41125dc11f" containerName="istio-proxy"
Sep 17 11:11:07.022111 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:07.022105    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="22799997-0d86-4ba2-abd8-9f41125dc11f" containerName="istio-proxy"
Sep 17 11:11:07.024956 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:07.024937    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7"
Sep 17 11:11:07.026964 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:07.026944    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:11:07.027059 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:07.026998    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:11:07.034120 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:07.034068    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7"]
Sep 17 11:11:07.073243 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:07.073208    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20f6fbbd-bdef-4479-98f7-39ba3e13fcb0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7\" (UID: \"20f6fbbd-bdef-4479-98f7-39ba3e13fcb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7"
Sep 17 11:11:07.143756 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:07.143725    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22799997-0d86-4ba2-abd8-9f41125dc11f" path="/var/lib/kubelet/pods/22799997-0d86-4ba2-abd8-9f41125dc11f/volumes"
Sep 17 11:11:07.174170 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:07.174132    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20f6fbbd-bdef-4479-98f7-39ba3e13fcb0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7\" (UID: \"20f6fbbd-bdef-4479-98f7-39ba3e13fcb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7"
Sep 17 11:11:07.174719 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:07.174696    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20f6fbbd-bdef-4479-98f7-39ba3e13fcb0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7\" (UID: \"20f6fbbd-bdef-4479-98f7-39ba3e13fcb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7"
Sep 17 11:11:07.337557 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:07.337474    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7"
Sep 17 11:11:07.616303 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:07.614065    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7"]
Sep 17 11:11:07.620290 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:11:07.620260    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f6fbbd_bdef_4479_98f7_39ba3e13fcb0.slice/crio-e94dd6c29076ca5704453bc01c5d3e7ff6341344e52efb4c470dbb920ee14dcc WatchSource:0}: Error finding container e94dd6c29076ca5704453bc01c5d3e7ff6341344e52efb4c470dbb920ee14dcc: Status 404 returned error can't find the container with id e94dd6c29076ca5704453bc01c5d3e7ff6341344e52efb4c470dbb920ee14dcc
Sep 17 11:11:07.684798 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:07.684757    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7" event={"ID":"20f6fbbd-bdef-4479-98f7-39ba3e13fcb0","Type":"ContainerStarted","Data":"e94dd6c29076ca5704453bc01c5d3e7ff6341344e52efb4c470dbb920ee14dcc"}
Sep 17 11:11:08.291454 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:11:08.291411    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f6fbbd_bdef_4479_98f7_39ba3e13fcb0.slice/crio-02b2b60d90cc2db75f444d2b759d3c79213ba977812a7ffd340ec0522f864321.scope\": RecentStats: unable to find data in memory cache]"
Sep 17 11:11:08.693758 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:08.693671    2564 generic.go:356] "Generic (PLEG): container finished" podID="20f6fbbd-bdef-4479-98f7-39ba3e13fcb0" containerID="02b2b60d90cc2db75f444d2b759d3c79213ba977812a7ffd340ec0522f864321" exitCode=0
Sep 17 11:11:08.693758 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:08.693726    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7" event={"ID":"20f6fbbd-bdef-4479-98f7-39ba3e13fcb0","Type":"ContainerDied","Data":"02b2b60d90cc2db75f444d2b759d3c79213ba977812a7ffd340ec0522f864321"}
Sep 17 11:11:09.835508 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:09.835477    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7"
Sep 17 11:11:09.899362 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:09.899332    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20f6fbbd-bdef-4479-98f7-39ba3e13fcb0-maas-api-service-ca\") pod \"20f6fbbd-bdef-4479-98f7-39ba3e13fcb0\" (UID: \"20f6fbbd-bdef-4479-98f7-39ba3e13fcb0\") "
Sep 17 11:11:09.899696 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:09.899674    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20f6fbbd-bdef-4479-98f7-39ba3e13fcb0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "20f6fbbd-bdef-4479-98f7-39ba3e13fcb0" (UID: "20f6fbbd-bdef-4479-98f7-39ba3e13fcb0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:11:10.000715 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:10.000681    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20f6fbbd-bdef-4479-98f7-39ba3e13fcb0-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:10.707018 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:10.706985    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7"
Sep 17 11:11:10.707189 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:10.706989    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7" event={"ID":"20f6fbbd-bdef-4479-98f7-39ba3e13fcb0","Type":"ContainerDied","Data":"e94dd6c29076ca5704453bc01c5d3e7ff6341344e52efb4c470dbb920ee14dcc"}
Sep 17 11:11:10.707189 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:10.707125    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e94dd6c29076ca5704453bc01c5d3e7ff6341344e52efb4c470dbb920ee14dcc"
Sep 17 11:11:11.143388 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:11.143359    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"]
Sep 17 11:11:11.143755 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:11.143736    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20f6fbbd-bdef-4479-98f7-39ba3e13fcb0" containerName="revoke-keys"
Sep 17 11:11:11.143755 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:11.143746    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="20f6fbbd-bdef-4479-98f7-39ba3e13fcb0" containerName="revoke-keys"
Sep 17 11:11:11.143830 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:11.143810    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="20f6fbbd-bdef-4479-98f7-39ba3e13fcb0" containerName="revoke-keys"
Sep 17 11:11:11.145763 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:11.145747    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"
Sep 17 11:11:11.147825 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:11.147802    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-15da354a\""
Sep 17 11:11:11.148100 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:11.148084    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-15da354a-dockercfg-zbv2j\""
Sep 17 11:11:11.158546 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:11.158522    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"]
Sep 17 11:11:11.211882 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:11.211844    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed5627f5-0089-455a-9e87-3b3956d271ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9\" (UID: \"ed5627f5-0089-455a-9e87-3b3956d271ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"
Sep 17 11:11:11.212104 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:11.211911    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmb8c\" (UniqueName: \"kubernetes.io/projected/ed5627f5-0089-455a-9e87-3b3956d271ef-kube-api-access-dmb8c\") pod \"payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9\" (UID: \"ed5627f5-0089-455a-9e87-3b3956d271ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"
Sep 17 11:11:11.316732 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:11.316665    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed5627f5-0089-455a-9e87-3b3956d271ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9\" (UID: \"ed5627f5-0089-455a-9e87-3b3956d271ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"
Sep 17 11:11:11.316732 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:11.316728    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmb8c\" (UniqueName: \"kubernetes.io/projected/ed5627f5-0089-455a-9e87-3b3956d271ef-kube-api-access-dmb8c\") pod \"payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9\" (UID: \"ed5627f5-0089-455a-9e87-3b3956d271ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"
Sep 17 11:11:11.317596 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:11.317568    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed5627f5-0089-455a-9e87-3b3956d271ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9\" (UID: \"ed5627f5-0089-455a-9e87-3b3956d271ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"
Sep 17 11:11:11.326409 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:11.326376    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmb8c\" (UniqueName: \"kubernetes.io/projected/ed5627f5-0089-455a-9e87-3b3956d271ef-kube-api-access-dmb8c\") pod \"payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9\" (UID: \"ed5627f5-0089-455a-9e87-3b3956d271ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"
Sep 17 11:11:11.457331 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:11.457244    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"
Sep 17 11:11:11.735920 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:11.735846    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"]
Sep 17 11:11:11.739106 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:11:11.739081    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded5627f5_0089_455a_9e87_3b3956d271ef.slice/crio-ca856925428466c32ad280ce707759cdbc2d089aae5b60954fc8e0610322f2b4 WatchSource:0}: Error finding container ca856925428466c32ad280ce707759cdbc2d089aae5b60954fc8e0610322f2b4: Status 404 returned error can't find the container with id ca856925428466c32ad280ce707759cdbc2d089aae5b60954fc8e0610322f2b4
Sep 17 11:11:12.726658 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:12.726619    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9" event={"ID":"ed5627f5-0089-455a-9e87-3b3956d271ef","Type":"ContainerStarted","Data":"fd62884b7ed9d08465fcf3e66f2957b9b940d39882174c9bedd52432c341d396"}
Sep 17 11:11:12.727077 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:12.726683    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9" event={"ID":"ed5627f5-0089-455a-9e87-3b3956d271ef","Type":"ContainerStarted","Data":"ca856925428466c32ad280ce707759cdbc2d089aae5b60954fc8e0610322f2b4"}
Sep 17 11:11:12.727077 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:12.726805    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"
Sep 17 11:11:12.755063 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:12.755015    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9" podStartSLOduration=1.755000391 podStartE2EDuration="1.755000391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:11:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:11:12.751531533 +0000 UTC m=+1068.067894688" watchObservedRunningTime="2026-09-17 11:11:12.755000391 +0000 UTC m=+1068.071363597"
Sep 17 11:11:18.377594 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.377521    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"]
Sep 17 11:11:18.378022 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.377747    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs" podUID="e159a98d-069b-4bf9-b8da-48e695d971f1" containerName="payload-processing" containerID="cri-o://1f5aef5b11496a1eb52909a3fa8642a2a8eb6de2e019050d05474395a985a408" gracePeriod=30
Sep 17 11:11:18.607565 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.607534    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7"]
Sep 17 11:11:18.610705 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.610678    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-79a70a-sqhd7"]
Sep 17 11:11:18.632241 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.632186    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"
Sep 17 11:11:18.698716 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.697222    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwwpw\" (UniqueName: \"kubernetes.io/projected/e159a98d-069b-4bf9-b8da-48e695d971f1-kube-api-access-hwwpw\") pod \"e159a98d-069b-4bf9-b8da-48e695d971f1\" (UID: \"e159a98d-069b-4bf9-b8da-48e695d971f1\") "
Sep 17 11:11:18.698716 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.697752    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e159a98d-069b-4bf9-b8da-48e695d971f1-plugins-config-volume\") pod \"e159a98d-069b-4bf9-b8da-48e695d971f1\" (UID: \"e159a98d-069b-4bf9-b8da-48e695d971f1\") "
Sep 17 11:11:18.698716 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.698221    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e159a98d-069b-4bf9-b8da-48e695d971f1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e159a98d-069b-4bf9-b8da-48e695d971f1" (UID: "e159a98d-069b-4bf9-b8da-48e695d971f1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:11:18.698716 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.698320    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e159a98d-069b-4bf9-b8da-48e695d971f1-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:18.703071 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.703027    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e159a98d-069b-4bf9-b8da-48e695d971f1-kube-api-access-hwwpw" (OuterVolumeSpecName: "kube-api-access-hwwpw") pod "e159a98d-069b-4bf9-b8da-48e695d971f1" (UID: "e159a98d-069b-4bf9-b8da-48e695d971f1"). InnerVolumeSpecName "kube-api-access-hwwpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:11:18.763546 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.763517    2564 generic.go:356] "Generic (PLEG): container finished" podID="e159a98d-069b-4bf9-b8da-48e695d971f1" containerID="1f5aef5b11496a1eb52909a3fa8642a2a8eb6de2e019050d05474395a985a408" exitCode=0
Sep 17 11:11:18.763698 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.763562    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"
Sep 17 11:11:18.763698 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.763570    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs" event={"ID":"e159a98d-069b-4bf9-b8da-48e695d971f1","Type":"ContainerDied","Data":"1f5aef5b11496a1eb52909a3fa8642a2a8eb6de2e019050d05474395a985a408"}
Sep 17 11:11:18.763698 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.763626    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs" event={"ID":"e159a98d-069b-4bf9-b8da-48e695d971f1","Type":"ContainerDied","Data":"77359dc3a0c349b63b0f97d524e1166262ef67725092035038a11815bc75353d"}
Sep 17 11:11:18.763698 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.763656    2564 scope.go:117] "RemoveContainer" containerID="1f5aef5b11496a1eb52909a3fa8642a2a8eb6de2e019050d05474395a985a408"
Sep 17 11:11:18.773262 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.773248    2564 scope.go:117] "RemoveContainer" containerID="1f5aef5b11496a1eb52909a3fa8642a2a8eb6de2e019050d05474395a985a408"
Sep 17 11:11:18.773520 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:11:18.773502    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f5aef5b11496a1eb52909a3fa8642a2a8eb6de2e019050d05474395a985a408\": container with ID starting with 1f5aef5b11496a1eb52909a3fa8642a2a8eb6de2e019050d05474395a985a408 not found: ID does not exist" containerID="1f5aef5b11496a1eb52909a3fa8642a2a8eb6de2e019050d05474395a985a408"
Sep 17 11:11:18.773588 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.773527    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f5aef5b11496a1eb52909a3fa8642a2a8eb6de2e019050d05474395a985a408"} err="failed to get container status \"1f5aef5b11496a1eb52909a3fa8642a2a8eb6de2e019050d05474395a985a408\": rpc error: code = NotFound desc = could not find container \"1f5aef5b11496a1eb52909a3fa8642a2a8eb6de2e019050d05474395a985a408\": container with ID starting with 1f5aef5b11496a1eb52909a3fa8642a2a8eb6de2e019050d05474395a985a408 not found: ID does not exist"
Sep 17 11:11:18.799450 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.799422    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwwpw\" (UniqueName: \"kubernetes.io/projected/e159a98d-069b-4bf9-b8da-48e695d971f1-kube-api-access-hwwpw\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:18.801764 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.801745    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"]
Sep 17 11:11:18.808279 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:18.808258    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-79a70a-6c6f4b7955-8rxcs"]
Sep 17 11:11:19.143200 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:19.143171    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20f6fbbd-bdef-4479-98f7-39ba3e13fcb0" path="/var/lib/kubelet/pods/20f6fbbd-bdef-4479-98f7-39ba3e13fcb0/volumes"
Sep 17 11:11:19.143543 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:19.143530    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e159a98d-069b-4bf9-b8da-48e695d971f1" path="/var/lib/kubelet/pods/e159a98d-069b-4bf9-b8da-48e695d971f1/volumes"
Sep 17 11:11:23.735293 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:23.735267    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"
Sep 17 11:11:25.977046 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:25.976997    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"]
Sep 17 11:11:25.978312 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:25.977769    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68" podUID="1016253a-7a65-4c96-ab4a-4b4524d207ca" containerName="istio-proxy" containerID="cri-o://bd369411caff1e75d6eb94582cc55ee3ba76023e434380c6f8be149298f065fd" gracePeriod=30
Sep 17 11:11:26.403690 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:26.403656    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf"]
Sep 17 11:11:26.404086 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:26.404072    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e159a98d-069b-4bf9-b8da-48e695d971f1" containerName="payload-processing"
Sep 17 11:11:26.404143 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:26.404087    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e159a98d-069b-4bf9-b8da-48e695d971f1" containerName="payload-processing"
Sep 17 11:11:26.404188 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:26.404166    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e159a98d-069b-4bf9-b8da-48e695d971f1" containerName="payload-processing"
Sep 17 11:11:26.413642 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:26.413616    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf"
Sep 17 11:11:26.415840 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:26.415817    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:11:26.416155 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:26.416141    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:11:26.422588 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:26.422566    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf"]
Sep 17 11:11:26.576730 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:26.576700    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb0b4456-abb0-4028-abc7-a673b59b846d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf\" (UID: \"eb0b4456-abb0-4028-abc7-a673b59b846d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf"
Sep 17 11:11:26.677850 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:26.677769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb0b4456-abb0-4028-abc7-a673b59b846d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf\" (UID: \"eb0b4456-abb0-4028-abc7-a673b59b846d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf"
Sep 17 11:11:26.678371 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:26.678349    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb0b4456-abb0-4028-abc7-a673b59b846d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf\" (UID: \"eb0b4456-abb0-4028-abc7-a673b59b846d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf"
Sep 17 11:11:26.725824 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:26.725794    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf"
Sep 17 11:11:26.817357 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:26.817328    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l"]
Sep 17 11:11:26.820616 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:26.820599    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l"
Sep 17 11:11:26.829558 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:26.829434    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l"]
Sep 17 11:11:26.982114 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:26.982041    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d18294d-ee16-4e49-abde-ca3dea34e00a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-15da354a-zpx9l\" (UID: \"4d18294d-ee16-4e49-abde-ca3dea34e00a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l"
Sep 17 11:11:27.004646 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:27.004620    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf"]
Sep 17 11:11:27.006090 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:11:27.006066    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb0b4456_abb0_4028_abc7_a673b59b846d.slice/crio-8c21c7a31f3a99272bcaeb49b3a074f74fdad50887e46f20f655f72da21294cf WatchSource:0}: Error finding container 8c21c7a31f3a99272bcaeb49b3a074f74fdad50887e46f20f655f72da21294cf: Status 404 returned error can't find the container with id 8c21c7a31f3a99272bcaeb49b3a074f74fdad50887e46f20f655f72da21294cf
Sep 17 11:11:27.083514 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:27.083485    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d18294d-ee16-4e49-abde-ca3dea34e00a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-15da354a-zpx9l\" (UID: \"4d18294d-ee16-4e49-abde-ca3dea34e00a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l"
Sep 17 11:11:27.084129 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:27.084111    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d18294d-ee16-4e49-abde-ca3dea34e00a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-15da354a-zpx9l\" (UID: \"4d18294d-ee16-4e49-abde-ca3dea34e00a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l"
Sep 17 11:11:27.134467 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:27.134440    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l"
Sep 17 11:11:27.410072 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:27.410044    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l"]
Sep 17 11:11:27.411009 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:11:27.410981    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d18294d_ee16_4e49_abde_ca3dea34e00a.slice/crio-f3db790ebcbc47336511b071308a77915e8ea7a4013c1a3e51623ee796cc0075 WatchSource:0}: Error finding container f3db790ebcbc47336511b071308a77915e8ea7a4013c1a3e51623ee796cc0075: Status 404 returned error can't find the container with id f3db790ebcbc47336511b071308a77915e8ea7a4013c1a3e51623ee796cc0075
Sep 17 11:11:27.826859 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:27.826827    2564 generic.go:356] "Generic (PLEG): container finished" podID="eb0b4456-abb0-4028-abc7-a673b59b846d" containerID="1b53531b22721fbc78635b1607fbaf8270d2947d31ba7738818d01f5932fd6de" exitCode=0
Sep 17 11:11:27.827018 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:27.826940    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf" event={"ID":"eb0b4456-abb0-4028-abc7-a673b59b846d","Type":"ContainerDied","Data":"1b53531b22721fbc78635b1607fbaf8270d2947d31ba7738818d01f5932fd6de"}
Sep 17 11:11:27.827018 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:27.826997    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf" event={"ID":"eb0b4456-abb0-4028-abc7-a673b59b846d","Type":"ContainerStarted","Data":"8c21c7a31f3a99272bcaeb49b3a074f74fdad50887e46f20f655f72da21294cf"}
Sep 17 11:11:27.828479 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:27.828459    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l" event={"ID":"4d18294d-ee16-4e49-abde-ca3dea34e00a","Type":"ContainerStarted","Data":"5fc949bb598ea57517c0e9aa656258151d941a43f4d44896c6de7fb45b1150da"}
Sep 17 11:11:27.828569 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:27.828506    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l" event={"ID":"4d18294d-ee16-4e49-abde-ca3dea34e00a","Type":"ContainerStarted","Data":"f3db790ebcbc47336511b071308a77915e8ea7a4013c1a3e51623ee796cc0075"}
Sep 17 11:11:27.870446 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:27.870369    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l" podStartSLOduration=1.8703558729999998 podStartE2EDuration="1.870355873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:11:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:11:27.867505671 +0000 UTC m=+1083.183868837" watchObservedRunningTime="2026-09-17 11:11:27.870355873 +0000 UTC m=+1083.186719028"
Sep 17 11:11:28.839509 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:28.839477    2564 generic.go:356] "Generic (PLEG): container finished" podID="4d18294d-ee16-4e49-abde-ca3dea34e00a" containerID="5fc949bb598ea57517c0e9aa656258151d941a43f4d44896c6de7fb45b1150da" exitCode=0
Sep 17 11:11:28.839999 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:28.839580    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l" event={"ID":"4d18294d-ee16-4e49-abde-ca3dea34e00a","Type":"ContainerDied","Data":"5fc949bb598ea57517c0e9aa656258151d941a43f4d44896c6de7fb45b1150da"}
Sep 17 11:11:28.984647 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:28.984622    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf"
Sep 17 11:11:29.105811 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:29.105726    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb0b4456-abb0-4028-abc7-a673b59b846d-maas-api-service-ca\") pod \"eb0b4456-abb0-4028-abc7-a673b59b846d\" (UID: \"eb0b4456-abb0-4028-abc7-a673b59b846d\") "
Sep 17 11:11:29.106124 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:29.106102    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb0b4456-abb0-4028-abc7-a673b59b846d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eb0b4456-abb0-4028-abc7-a673b59b846d" (UID: "eb0b4456-abb0-4028-abc7-a673b59b846d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:11:29.195760 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:29.195722    2564 patch_prober.go:28] interesting pod/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 17 11:11:29.195976 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:29.195782    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68" podUID="1016253a-7a65-4c96-ab4a-4b4524d207ca" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 11:11:29.207562 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:29.207535    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb0b4456-abb0-4028-abc7-a673b59b846d-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:29.849193 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:29.849159    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf"
Sep 17 11:11:29.849193 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:29.849179    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf" event={"ID":"eb0b4456-abb0-4028-abc7-a673b59b846d","Type":"ContainerDied","Data":"8c21c7a31f3a99272bcaeb49b3a074f74fdad50887e46f20f655f72da21294cf"}
Sep 17 11:11:29.849713 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:29.849227    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c21c7a31f3a99272bcaeb49b3a074f74fdad50887e46f20f655f72da21294cf"
Sep 17 11:11:29.994379 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:29.994356    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l"
Sep 17 11:11:30.118985 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:30.118869    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d18294d-ee16-4e49-abde-ca3dea34e00a-maas-api-service-ca\") pod \"4d18294d-ee16-4e49-abde-ca3dea34e00a\" (UID: \"4d18294d-ee16-4e49-abde-ca3dea34e00a\") "
Sep 17 11:11:30.119235 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:30.119211    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d18294d-ee16-4e49-abde-ca3dea34e00a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4d18294d-ee16-4e49-abde-ca3dea34e00a" (UID: "4d18294d-ee16-4e49-abde-ca3dea34e00a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:11:30.136999 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:30.136967    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:11:30.220543 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:30.220504    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d18294d-ee16-4e49-abde-ca3dea34e00a-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:30.856364 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:30.856323    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l" event={"ID":"4d18294d-ee16-4e49-abde-ca3dea34e00a","Type":"ContainerDied","Data":"f3db790ebcbc47336511b071308a77915e8ea7a4013c1a3e51623ee796cc0075"}
Sep 17 11:11:30.856364 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:30.856358    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l"
Sep 17 11:11:30.856847 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:30.856385    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3db790ebcbc47336511b071308a77915e8ea7a4013c1a3e51623ee796cc0075"
Sep 17 11:11:31.273618 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.273597    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:11:31.432464 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.432364    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-workload-socket\") pod \"1016253a-7a65-4c96-ab4a-4b4524d207ca\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") "
Sep 17 11:11:31.432464 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.432415    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1016253a-7a65-4c96-ab4a-4b4524d207ca-istiod-ca-cert\") pod \"1016253a-7a65-4c96-ab4a-4b4524d207ca\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") "
Sep 17 11:11:31.432464 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.432455    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-podinfo\") pod \"1016253a-7a65-4c96-ab4a-4b4524d207ca\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") "
Sep 17 11:11:31.432756 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.432480    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-credential-socket\") pod \"1016253a-7a65-4c96-ab4a-4b4524d207ca\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") "
Sep 17 11:11:31.432756 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.432508    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-workload-certs\") pod \"1016253a-7a65-4c96-ab4a-4b4524d207ca\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") "
Sep 17 11:11:31.432756 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.432606    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-envoy\") pod \"1016253a-7a65-4c96-ab4a-4b4524d207ca\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") "
Sep 17 11:11:31.432756 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.432604    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1016253a-7a65-4c96-ab4a-4b4524d207ca" (UID: "1016253a-7a65-4c96-ab4a-4b4524d207ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:11:31.432756 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.432656    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-data\") pod \"1016253a-7a65-4c96-ab4a-4b4524d207ca\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") "
Sep 17 11:11:31.432756 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.432691    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-token\") pod \"1016253a-7a65-4c96-ab4a-4b4524d207ca\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") "
Sep 17 11:11:31.432756 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.432723    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-782fq\" (UniqueName: \"kubernetes.io/projected/1016253a-7a65-4c96-ab4a-4b4524d207ca-kube-api-access-782fq\") pod \"1016253a-7a65-4c96-ab4a-4b4524d207ca\" (UID: \"1016253a-7a65-4c96-ab4a-4b4524d207ca\") "
Sep 17 11:11:31.432756 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.432739    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1016253a-7a65-4c96-ab4a-4b4524d207ca" (UID: "1016253a-7a65-4c96-ab4a-4b4524d207ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:11:31.433242 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.432795    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1016253a-7a65-4c96-ab4a-4b4524d207ca" (UID: "1016253a-7a65-4c96-ab4a-4b4524d207ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:11:31.433242 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.432890    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-data" (OuterVolumeSpecName: "istio-data") pod "1016253a-7a65-4c96-ab4a-4b4524d207ca" (UID: "1016253a-7a65-4c96-ab4a-4b4524d207ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:11:31.433242 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.432986    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1016253a-7a65-4c96-ab4a-4b4524d207ca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1016253a-7a65-4c96-ab4a-4b4524d207ca" (UID: "1016253a-7a65-4c96-ab4a-4b4524d207ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:11:31.433242 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.433213    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-workload-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:31.433242 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.433229    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1016253a-7a65-4c96-ab4a-4b4524d207ca-istiod-ca-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:31.433242 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.433238    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-credential-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:31.433242 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.433247    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-workload-certs\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:31.433770 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.433256    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-data\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:31.434881 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.434853    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1016253a-7a65-4c96-ab4a-4b4524d207ca" (UID: "1016253a-7a65-4c96-ab4a-4b4524d207ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 11:11:31.435182 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.435156    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1016253a-7a65-4c96-ab4a-4b4524d207ca-kube-api-access-782fq" (OuterVolumeSpecName: "kube-api-access-782fq") pod "1016253a-7a65-4c96-ab4a-4b4524d207ca" (UID: "1016253a-7a65-4c96-ab4a-4b4524d207ca"). InnerVolumeSpecName "kube-api-access-782fq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:11:31.435349 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.435323    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-token" (OuterVolumeSpecName: "istio-token") pod "1016253a-7a65-4c96-ab4a-4b4524d207ca" (UID: "1016253a-7a65-4c96-ab4a-4b4524d207ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:11:31.435426 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.435371    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1016253a-7a65-4c96-ab4a-4b4524d207ca" (UID: "1016253a-7a65-4c96-ab4a-4b4524d207ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:11:31.534044 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.534006    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-envoy\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:31.534044 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.534034    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-token\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:31.534044 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.534043    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-782fq\" (UniqueName: \"kubernetes.io/projected/1016253a-7a65-4c96-ab4a-4b4524d207ca-kube-api-access-782fq\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:31.534044 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.534052    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1016253a-7a65-4c96-ab4a-4b4524d207ca-istio-podinfo\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:31.863451 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.863420    2564 generic.go:356] "Generic (PLEG): container finished" podID="1016253a-7a65-4c96-ab4a-4b4524d207ca" containerID="bd369411caff1e75d6eb94582cc55ee3ba76023e434380c6f8be149298f065fd" exitCode=0
Sep 17 11:11:31.863824 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.863470    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"
Sep 17 11:11:31.863824 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.863512    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68" event={"ID":"1016253a-7a65-4c96-ab4a-4b4524d207ca","Type":"ContainerDied","Data":"bd369411caff1e75d6eb94582cc55ee3ba76023e434380c6f8be149298f065fd"}
Sep 17 11:11:31.863824 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.863569    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68" event={"ID":"1016253a-7a65-4c96-ab4a-4b4524d207ca","Type":"ContainerDied","Data":"783f25bc58c566ab1cff7bf611c2351ee1a6e2c03706362f22d7a944be860886"}
Sep 17 11:11:31.863824 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.863600    2564 scope.go:117] "RemoveContainer" containerID="bd369411caff1e75d6eb94582cc55ee3ba76023e434380c6f8be149298f065fd"
Sep 17 11:11:31.874220 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.874203    2564 scope.go:117] "RemoveContainer" containerID="8df248d75d6a04064c9fc2b4cdd091df928f665000ef4d8b6aa590b47b901f48"
Sep 17 11:11:31.883135 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.883115    2564 scope.go:117] "RemoveContainer" containerID="bd369411caff1e75d6eb94582cc55ee3ba76023e434380c6f8be149298f065fd"
Sep 17 11:11:31.883394 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:11:31.883375    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd369411caff1e75d6eb94582cc55ee3ba76023e434380c6f8be149298f065fd\": container with ID starting with bd369411caff1e75d6eb94582cc55ee3ba76023e434380c6f8be149298f065fd not found: ID does not exist" containerID="bd369411caff1e75d6eb94582cc55ee3ba76023e434380c6f8be149298f065fd"
Sep 17 11:11:31.883430 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.883404    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd369411caff1e75d6eb94582cc55ee3ba76023e434380c6f8be149298f065fd"} err="failed to get container status \"bd369411caff1e75d6eb94582cc55ee3ba76023e434380c6f8be149298f065fd\": rpc error: code = NotFound desc = could not find container \"bd369411caff1e75d6eb94582cc55ee3ba76023e434380c6f8be149298f065fd\": container with ID starting with bd369411caff1e75d6eb94582cc55ee3ba76023e434380c6f8be149298f065fd not found: ID does not exist"
Sep 17 11:11:31.883430 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.883422    2564 scope.go:117] "RemoveContainer" containerID="8df248d75d6a04064c9fc2b4cdd091df928f665000ef4d8b6aa590b47b901f48"
Sep 17 11:11:31.883684 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:11:31.883667    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8df248d75d6a04064c9fc2b4cdd091df928f665000ef4d8b6aa590b47b901f48\": container with ID starting with 8df248d75d6a04064c9fc2b4cdd091df928f665000ef4d8b6aa590b47b901f48 not found: ID does not exist" containerID="8df248d75d6a04064c9fc2b4cdd091df928f665000ef4d8b6aa590b47b901f48"
Sep 17 11:11:31.883731 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.883690    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8df248d75d6a04064c9fc2b4cdd091df928f665000ef4d8b6aa590b47b901f48"} err="failed to get container status \"8df248d75d6a04064c9fc2b4cdd091df928f665000ef4d8b6aa590b47b901f48\": rpc error: code = NotFound desc = could not find container \"8df248d75d6a04064c9fc2b4cdd091df928f665000ef4d8b6aa590b47b901f48\": container with ID starting with 8df248d75d6a04064c9fc2b4cdd091df928f665000ef4d8b6aa590b47b901f48 not found: ID does not exist"
Sep 17 11:11:31.903210 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.903182    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"]
Sep 17 11:11:31.907782 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:31.907749    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-79a70a-openshift-default-6c6b698c68-ghb68"]
Sep 17 11:11:33.143846 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:33.143812    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1016253a-7a65-4c96-ab4a-4b4524d207ca" path="/var/lib/kubelet/pods/1016253a-7a65-4c96-ab4a-4b4524d207ca/volumes"
Sep 17 11:11:37.555280 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:37.555241    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"]
Sep 17 11:11:37.555669 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:37.555552    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9" podUID="ed5627f5-0089-455a-9e87-3b3956d271ef" containerName="payload-pre-processing" containerID="cri-o://fd62884b7ed9d08465fcf3e66f2957b9b940d39882174c9bedd52432c341d396" gracePeriod=30
Sep 17 11:11:37.807888 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:37.807800    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l"]
Sep 17 11:11:37.810370 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:37.810343    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-15da354a-zpx9l"]
Sep 17 11:11:37.842630 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:37.842602    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"
Sep 17 11:11:37.899960 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:37.899873    2564 generic.go:356] "Generic (PLEG): container finished" podID="ed5627f5-0089-455a-9e87-3b3956d271ef" containerID="fd62884b7ed9d08465fcf3e66f2957b9b940d39882174c9bedd52432c341d396" exitCode=0
Sep 17 11:11:37.899960 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:37.899942    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"
Sep 17 11:11:37.900208 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:37.899990    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9" event={"ID":"ed5627f5-0089-455a-9e87-3b3956d271ef","Type":"ContainerDied","Data":"fd62884b7ed9d08465fcf3e66f2957b9b940d39882174c9bedd52432c341d396"}
Sep 17 11:11:37.900208 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:37.900065    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9" event={"ID":"ed5627f5-0089-455a-9e87-3b3956d271ef","Type":"ContainerDied","Data":"ca856925428466c32ad280ce707759cdbc2d089aae5b60954fc8e0610322f2b4"}
Sep 17 11:11:37.900208 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:37.900099    2564 scope.go:117] "RemoveContainer" containerID="fd62884b7ed9d08465fcf3e66f2957b9b940d39882174c9bedd52432c341d396"
Sep 17 11:11:37.909478 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:37.909457    2564 scope.go:117] "RemoveContainer" containerID="fd62884b7ed9d08465fcf3e66f2957b9b940d39882174c9bedd52432c341d396"
Sep 17 11:11:37.909818 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:11:37.909797    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd62884b7ed9d08465fcf3e66f2957b9b940d39882174c9bedd52432c341d396\": container with ID starting with fd62884b7ed9d08465fcf3e66f2957b9b940d39882174c9bedd52432c341d396 not found: ID does not exist" containerID="fd62884b7ed9d08465fcf3e66f2957b9b940d39882174c9bedd52432c341d396"
Sep 17 11:11:37.909871 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:37.909829    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd62884b7ed9d08465fcf3e66f2957b9b940d39882174c9bedd52432c341d396"} err="failed to get container status \"fd62884b7ed9d08465fcf3e66f2957b9b940d39882174c9bedd52432c341d396\": rpc error: code = NotFound desc = could not find container \"fd62884b7ed9d08465fcf3e66f2957b9b940d39882174c9bedd52432c341d396\": container with ID starting with fd62884b7ed9d08465fcf3e66f2957b9b940d39882174c9bedd52432c341d396 not found: ID does not exist"
Sep 17 11:11:38.000675 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:38.000638    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed5627f5-0089-455a-9e87-3b3956d271ef-plugins-config-volume\") pod \"ed5627f5-0089-455a-9e87-3b3956d271ef\" (UID: \"ed5627f5-0089-455a-9e87-3b3956d271ef\") "
Sep 17 11:11:38.000884 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:38.000738    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmb8c\" (UniqueName: \"kubernetes.io/projected/ed5627f5-0089-455a-9e87-3b3956d271ef-kube-api-access-dmb8c\") pod \"ed5627f5-0089-455a-9e87-3b3956d271ef\" (UID: \"ed5627f5-0089-455a-9e87-3b3956d271ef\") "
Sep 17 11:11:38.001194 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:38.001161    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed5627f5-0089-455a-9e87-3b3956d271ef-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ed5627f5-0089-455a-9e87-3b3956d271ef" (UID: "ed5627f5-0089-455a-9e87-3b3956d271ef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:11:38.003402 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:38.003369    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed5627f5-0089-455a-9e87-3b3956d271ef-kube-api-access-dmb8c" (OuterVolumeSpecName: "kube-api-access-dmb8c") pod "ed5627f5-0089-455a-9e87-3b3956d271ef" (UID: "ed5627f5-0089-455a-9e87-3b3956d271ef"). InnerVolumeSpecName "kube-api-access-dmb8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:11:38.102269 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:38.102173    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed5627f5-0089-455a-9e87-3b3956d271ef-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:38.102269 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:38.102213    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmb8c\" (UniqueName: \"kubernetes.io/projected/ed5627f5-0089-455a-9e87-3b3956d271ef-kube-api-access-dmb8c\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:38.242232 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:38.242194    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"]
Sep 17 11:11:38.250569 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:38.250543    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-15da354a-5444c7d666-5pvv9"]
Sep 17 11:11:38.343327 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:38.343292    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf"]
Sep 17 11:11:38.356012 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:38.355930    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10e0b-rp8zf"]
Sep 17 11:11:39.010490 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.010454    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"]
Sep 17 11:11:39.011020 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.010710    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8" podUID="3742dddb-316d-4bfe-9bb6-5327545bafaa" containerName="istio-proxy" containerID="cri-o://40d3554005f884ed2f13cfca6247a23cdea499c15ead918f3428584b3d7b328b" gracePeriod=30
Sep 17 11:11:39.144187 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.144157    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d18294d-ee16-4e49-abde-ca3dea34e00a" path="/var/lib/kubelet/pods/4d18294d-ee16-4e49-abde-ca3dea34e00a/volumes"
Sep 17 11:11:39.165764 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.144653    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb0b4456-abb0-4028-abc7-a673b59b846d" path="/var/lib/kubelet/pods/eb0b4456-abb0-4028-abc7-a673b59b846d/volumes"
Sep 17 11:11:39.165764 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.145217    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed5627f5-0089-455a-9e87-3b3956d271ef" path="/var/lib/kubelet/pods/ed5627f5-0089-455a-9e87-3b3956d271ef/volumes"
Sep 17 11:11:39.472429 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.472405    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:39.615566 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.615484    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-envoy\") pod \"3742dddb-316d-4bfe-9bb6-5327545bafaa\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") "
Sep 17 11:11:39.615566 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.615541    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-workload-certs\") pod \"3742dddb-316d-4bfe-9bb6-5327545bafaa\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") "
Sep 17 11:11:39.615768 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.615572    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3742dddb-316d-4bfe-9bb6-5327545bafaa-istiod-ca-cert\") pod \"3742dddb-316d-4bfe-9bb6-5327545bafaa\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") "
Sep 17 11:11:39.615768 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.615594    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-podinfo\") pod \"3742dddb-316d-4bfe-9bb6-5327545bafaa\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") "
Sep 17 11:11:39.615768 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.615625    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfkh8\" (UniqueName: \"kubernetes.io/projected/3742dddb-316d-4bfe-9bb6-5327545bafaa-kube-api-access-dfkh8\") pod \"3742dddb-316d-4bfe-9bb6-5327545bafaa\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") "
Sep 17 11:11:39.615971 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.615857    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-credential-socket\") pod \"3742dddb-316d-4bfe-9bb6-5327545bafaa\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") "
Sep 17 11:11:39.615971 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.615865    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3742dddb-316d-4bfe-9bb6-5327545bafaa" (UID: "3742dddb-316d-4bfe-9bb6-5327545bafaa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:11:39.615971 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.615932    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-workload-socket\") pod \"3742dddb-316d-4bfe-9bb6-5327545bafaa\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") "
Sep 17 11:11:39.616132 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.616014    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-token\") pod \"3742dddb-316d-4bfe-9bb6-5327545bafaa\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") "
Sep 17 11:11:39.616132 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.616020    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3742dddb-316d-4bfe-9bb6-5327545bafaa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3742dddb-316d-4bfe-9bb6-5327545bafaa" (UID: "3742dddb-316d-4bfe-9bb6-5327545bafaa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:11:39.616132 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.616079    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-data\") pod \"3742dddb-316d-4bfe-9bb6-5327545bafaa\" (UID: \"3742dddb-316d-4bfe-9bb6-5327545bafaa\") "
Sep 17 11:11:39.616132 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.616090    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3742dddb-316d-4bfe-9bb6-5327545bafaa" (UID: "3742dddb-316d-4bfe-9bb6-5327545bafaa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:11:39.616352 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.616141    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3742dddb-316d-4bfe-9bb6-5327545bafaa" (UID: "3742dddb-316d-4bfe-9bb6-5327545bafaa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:11:39.616576 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.616553    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-data" (OuterVolumeSpecName: "istio-data") pod "3742dddb-316d-4bfe-9bb6-5327545bafaa" (UID: "3742dddb-316d-4bfe-9bb6-5327545bafaa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:11:39.616680 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.616660    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3742dddb-316d-4bfe-9bb6-5327545bafaa-istiod-ca-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:39.616750 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.616686    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-credential-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:39.616750 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.616701    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-workload-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:39.616750 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.616716    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-workload-certs\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:39.618118 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.618085    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3742dddb-316d-4bfe-9bb6-5327545bafaa" (UID: "3742dddb-316d-4bfe-9bb6-5327545bafaa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:11:39.618118 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.618100    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3742dddb-316d-4bfe-9bb6-5327545bafaa-kube-api-access-dfkh8" (OuterVolumeSpecName: "kube-api-access-dfkh8") pod "3742dddb-316d-4bfe-9bb6-5327545bafaa" (UID: "3742dddb-316d-4bfe-9bb6-5327545bafaa"). InnerVolumeSpecName "kube-api-access-dfkh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:11:39.618298 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.618176    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3742dddb-316d-4bfe-9bb6-5327545bafaa" (UID: "3742dddb-316d-4bfe-9bb6-5327545bafaa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 11:11:39.618485 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.618467    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-token" (OuterVolumeSpecName: "istio-token") pod "3742dddb-316d-4bfe-9bb6-5327545bafaa" (UID: "3742dddb-316d-4bfe-9bb6-5327545bafaa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:11:39.717935 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.717877    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfkh8\" (UniqueName: \"kubernetes.io/projected/3742dddb-316d-4bfe-9bb6-5327545bafaa-kube-api-access-dfkh8\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:39.717935 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.717934    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-token\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:39.717935 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.717944    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-data\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:39.717935 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.717951    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-envoy\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:39.718215 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.717958    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3742dddb-316d-4bfe-9bb6-5327545bafaa-istio-podinfo\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:11:39.803788 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.803755    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"]
Sep 17 11:11:39.804206 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804193    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed5627f5-0089-455a-9e87-3b3956d271ef" containerName="payload-pre-processing"
Sep 17 11:11:39.804206 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804208    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed5627f5-0089-455a-9e87-3b3956d271ef" containerName="payload-pre-processing"
Sep 17 11:11:39.804325 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804216    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d18294d-ee16-4e49-abde-ca3dea34e00a" containerName="revoke-keys"
Sep 17 11:11:39.804325 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804221    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d18294d-ee16-4e49-abde-ca3dea34e00a" containerName="revoke-keys"
Sep 17 11:11:39.804325 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804239    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb0b4456-abb0-4028-abc7-a673b59b846d" containerName="revoke-keys"
Sep 17 11:11:39.804325 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804245    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb0b4456-abb0-4028-abc7-a673b59b846d" containerName="revoke-keys"
Sep 17 11:11:39.804325 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804257    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1016253a-7a65-4c96-ab4a-4b4524d207ca" containerName="istio-proxy"
Sep 17 11:11:39.804325 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804262    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1016253a-7a65-4c96-ab4a-4b4524d207ca" containerName="istio-proxy"
Sep 17 11:11:39.804325 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804269    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3742dddb-316d-4bfe-9bb6-5327545bafaa" containerName="istio-proxy"
Sep 17 11:11:39.804325 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804274    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3742dddb-316d-4bfe-9bb6-5327545bafaa" containerName="istio-proxy"
Sep 17 11:11:39.804325 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804281    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1016253a-7a65-4c96-ab4a-4b4524d207ca" containerName="istio-proxy"
Sep 17 11:11:39.804325 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804286    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1016253a-7a65-4c96-ab4a-4b4524d207ca" containerName="istio-proxy"
Sep 17 11:11:39.804659 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804341    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d18294d-ee16-4e49-abde-ca3dea34e00a" containerName="revoke-keys"
Sep 17 11:11:39.804659 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804349    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1016253a-7a65-4c96-ab4a-4b4524d207ca" containerName="istio-proxy"
Sep 17 11:11:39.804659 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804359    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3742dddb-316d-4bfe-9bb6-5327545bafaa" containerName="istio-proxy"
Sep 17 11:11:39.804659 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804365    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed5627f5-0089-455a-9e87-3b3956d271ef" containerName="payload-pre-processing"
Sep 17 11:11:39.804659 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804370    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb0b4456-abb0-4028-abc7-a673b59b846d" containerName="revoke-keys"
Sep 17 11:11:39.804659 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.804375    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1016253a-7a65-4c96-ab4a-4b4524d207ca" containerName="istio-proxy"
Sep 17 11:11:39.806595 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.806575    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.808580 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.808560    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-15f9c946-openshift-default-dockercfg-2j6bh\""
Sep 17 11:11:39.819157 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.819132    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95ddf315-2321-4b90-9c13-00f0a603f109-istio-podinfo\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.819265 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.819171    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-istio-envoy\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.819265 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.819236    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95ddf315-2321-4b90-9c13-00f0a603f109-istiod-ca-cert\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.819265 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.819254    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-workload-socket\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.819472 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.819277    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-workload-certs\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.819472 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.819292    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95ddf315-2321-4b90-9c13-00f0a603f109-istio-token\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.819472 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.819320    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-credential-socket\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.819472 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.819395    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbfh2\" (UniqueName: \"kubernetes.io/projected/95ddf315-2321-4b90-9c13-00f0a603f109-kube-api-access-bbfh2\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.819653 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.819480    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-istio-data\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.823934 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.823879    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"]
Sep 17 11:11:39.916922 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.916783    2564 generic.go:356] "Generic (PLEG): container finished" podID="3742dddb-316d-4bfe-9bb6-5327545bafaa" containerID="40d3554005f884ed2f13cfca6247a23cdea499c15ead918f3428584b3d7b328b" exitCode=0
Sep 17 11:11:39.917091 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.917020    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8" event={"ID":"3742dddb-316d-4bfe-9bb6-5327545bafaa","Type":"ContainerDied","Data":"40d3554005f884ed2f13cfca6247a23cdea499c15ead918f3428584b3d7b328b"}
Sep 17 11:11:39.917091 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.917078    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8" event={"ID":"3742dddb-316d-4bfe-9bb6-5327545bafaa","Type":"ContainerDied","Data":"011cec1f6702b45a751d80ae6dd7f083336acd30ee76f2a2955897f021e6beb3"}
Sep 17 11:11:39.917211 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.917110    2564 scope.go:117] "RemoveContainer" containerID="40d3554005f884ed2f13cfca6247a23cdea499c15ead918f3428584b3d7b328b"
Sep 17 11:11:39.917314 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.917284    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"
Sep 17 11:11:39.920257 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.920220    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-istio-envoy\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.920336 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.920285    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95ddf315-2321-4b90-9c13-00f0a603f109-istiod-ca-cert\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.920336 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.920312    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-workload-socket\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.921042 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.920646    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-workload-certs\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.921042 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.920722    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-workload-socket\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.921042 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.921041    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95ddf315-2321-4b90-9c13-00f0a603f109-istio-token\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.921042 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.920975    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-workload-certs\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.921042 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.921103    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-credential-socket\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.921520 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.921139    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbfh2\" (UniqueName: \"kubernetes.io/projected/95ddf315-2321-4b90-9c13-00f0a603f109-kube-api-access-bbfh2\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.921520 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.921206    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-istio-data\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.921520 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.921229    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95ddf315-2321-4b90-9c13-00f0a603f109-istiod-ca-cert\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.921520 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.921249    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95ddf315-2321-4b90-9c13-00f0a603f109-istio-podinfo\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.921520 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.921407    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-credential-socket\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.921781 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.921634    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-istio-data\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.923069 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.923046    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-istio-envoy\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.923976 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.923958    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95ddf315-2321-4b90-9c13-00f0a603f109-istio-podinfo\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.930656 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.930634    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95ddf315-2321-4b90-9c13-00f0a603f109-istio-token\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.930931 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.930881    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbfh2\" (UniqueName: \"kubernetes.io/projected/95ddf315-2321-4b90-9c13-00f0a603f109-kube-api-access-bbfh2\") pod \"e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") " pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:39.932565 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.932542    2564 scope.go:117] "RemoveContainer" containerID="40d3554005f884ed2f13cfca6247a23cdea499c15ead918f3428584b3d7b328b"
Sep 17 11:11:39.933005 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:11:39.932943    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40d3554005f884ed2f13cfca6247a23cdea499c15ead918f3428584b3d7b328b\": container with ID starting with 40d3554005f884ed2f13cfca6247a23cdea499c15ead918f3428584b3d7b328b not found: ID does not exist" containerID="40d3554005f884ed2f13cfca6247a23cdea499c15ead918f3428584b3d7b328b"
Sep 17 11:11:39.933005 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.932972    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40d3554005f884ed2f13cfca6247a23cdea499c15ead918f3428584b3d7b328b"} err="failed to get container status \"40d3554005f884ed2f13cfca6247a23cdea499c15ead918f3428584b3d7b328b\": rpc error: code = NotFound desc = could not find container \"40d3554005f884ed2f13cfca6247a23cdea499c15ead918f3428584b3d7b328b\": container with ID starting with 40d3554005f884ed2f13cfca6247a23cdea499c15ead918f3428584b3d7b328b not found: ID does not exist"
Sep 17 11:11:39.946756 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.946726    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"]
Sep 17 11:11:39.952791 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:39.952762    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-15da354a-openshift-default-5f7499488-4jfc8"]
Sep 17 11:11:40.117194 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:40.117156    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:40.403846 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:40.403823    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"]
Sep 17 11:11:40.405548 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:11:40.405514    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95ddf315_2321_4b90_9c13_00f0a603f109.slice/crio-f652756a58372df437728a3fe193cc4d2c2678441b66831b478f283edb6a81f8 WatchSource:0}: Error finding container f652756a58372df437728a3fe193cc4d2c2678441b66831b478f283edb6a81f8: Status 404 returned error can't find the container with id f652756a58372df437728a3fe193cc4d2c2678441b66831b478f283edb6a81f8
Sep 17 11:11:40.407783 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:40.407753    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:11:40.407852 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:40.407816    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:11:40.407893 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:40.407856    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:11:40.925447 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:40.925411    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn" event={"ID":"95ddf315-2321-4b90-9c13-00f0a603f109","Type":"ContainerStarted","Data":"1bd51abcaad85bda1155cae3ece13f0a28368d23bca096ec71465d650b54c74b"}
Sep 17 11:11:40.925623 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:40.925462    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn" event={"ID":"95ddf315-2321-4b90-9c13-00f0a603f109","Type":"ContainerStarted","Data":"f652756a58372df437728a3fe193cc4d2c2678441b66831b478f283edb6a81f8"}
Sep 17 11:11:40.949767 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:40.949719    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn" podStartSLOduration=1.949703579 podStartE2EDuration="1.949703579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:11:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:11:40.949603333 +0000 UTC m=+1096.265966489" watchObservedRunningTime="2026-09-17 11:11:40.949703579 +0000 UTC m=+1096.266066827"
Sep 17 11:11:41.117669 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:41.117631    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:41.123488 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:41.123463    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:41.143051 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:41.143018    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3742dddb-316d-4bfe-9bb6-5327545bafaa" path="/var/lib/kubelet/pods/3742dddb-316d-4bfe-9bb6-5327545bafaa/volumes"
Sep 17 11:11:41.933646 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:41.933613    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:41.934617 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:41.934593    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:11:44.979814 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:44.979779    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"]
Sep 17 11:11:44.982407 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:44.982391    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"
Sep 17 11:11:44.984759 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:44.984740    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-15f9c946-dockercfg-qhz92\""
Sep 17 11:11:44.984759 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:44.984750    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-15f9c946\""
Sep 17 11:11:44.996427 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:44.996400    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"]
Sep 17 11:11:45.077417 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:45.077383    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55cs8\" (UniqueName: \"kubernetes.io/projected/5e58a14e-4e30-4b6e-852d-01ecfb4f3a04-kube-api-access-55cs8\") pod \"payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4\" (UID: \"5e58a14e-4e30-4b6e-852d-01ecfb4f3a04\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"
Sep 17 11:11:45.077588 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:45.077454    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e58a14e-4e30-4b6e-852d-01ecfb4f3a04-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4\" (UID: \"5e58a14e-4e30-4b6e-852d-01ecfb4f3a04\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"
Sep 17 11:11:45.178296 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:45.178258    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55cs8\" (UniqueName: \"kubernetes.io/projected/5e58a14e-4e30-4b6e-852d-01ecfb4f3a04-kube-api-access-55cs8\") pod \"payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4\" (UID: \"5e58a14e-4e30-4b6e-852d-01ecfb4f3a04\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"
Sep 17 11:11:45.178470 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:45.178343    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e58a14e-4e30-4b6e-852d-01ecfb4f3a04-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4\" (UID: \"5e58a14e-4e30-4b6e-852d-01ecfb4f3a04\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"
Sep 17 11:11:45.179173 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:45.179154    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e58a14e-4e30-4b6e-852d-01ecfb4f3a04-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4\" (UID: \"5e58a14e-4e30-4b6e-852d-01ecfb4f3a04\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"
Sep 17 11:11:45.187602 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:45.187577    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55cs8\" (UniqueName: \"kubernetes.io/projected/5e58a14e-4e30-4b6e-852d-01ecfb4f3a04-kube-api-access-55cs8\") pod \"payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4\" (UID: \"5e58a14e-4e30-4b6e-852d-01ecfb4f3a04\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"
Sep 17 11:11:45.293055 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:45.293028    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"
Sep 17 11:11:45.596987 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:45.596960    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"]
Sep 17 11:11:45.598324 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:11:45.598294    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e58a14e_4e30_4b6e_852d_01ecfb4f3a04.slice/crio-ca8fe0b641d0db3d5d38c9b4bb80d399fc514646bbcba6026d264fa423fa94ac WatchSource:0}: Error finding container ca8fe0b641d0db3d5d38c9b4bb80d399fc514646bbcba6026d264fa423fa94ac: Status 404 returned error can't find the container with id ca8fe0b641d0db3d5d38c9b4bb80d399fc514646bbcba6026d264fa423fa94ac
Sep 17 11:11:45.968175 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:45.968073    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4" event={"ID":"5e58a14e-4e30-4b6e-852d-01ecfb4f3a04","Type":"ContainerStarted","Data":"d243acc7656e85c86cd1ea74998b4365497221a392db8df5253fe570a816fda2"}
Sep 17 11:11:45.968328 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:45.968176    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4" event={"ID":"5e58a14e-4e30-4b6e-852d-01ecfb4f3a04","Type":"ContainerStarted","Data":"ca8fe0b641d0db3d5d38c9b4bb80d399fc514646bbcba6026d264fa423fa94ac"}
Sep 17 11:11:45.968328 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:45.968209    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"
Sep 17 11:11:45.997131 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:45.997073    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4" podStartSLOduration=1.9970550299999998 podStartE2EDuration="1.99705503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:11:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:11:45.993582414 +0000 UTC m=+1101.309945569" watchObservedRunningTime="2026-09-17 11:11:45.99705503 +0000 UTC m=+1101.313418185"
Sep 17 11:11:47.269433 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.269400    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"]
Sep 17 11:11:47.272020 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.272002    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.279728 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.279703    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-026f86-openshift-default-dockercfg-rwlms\""
Sep 17 11:11:47.299576 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.299551    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be46ac4-4bc1-4975-9223-791bd5028769-istio-podinfo\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.299687 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.299653    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-istio-envoy\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.299752 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.299683    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-workload-socket\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.299807 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.299770    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be46ac4-4bc1-4975-9223-791bd5028769-istiod-ca-cert\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.299807 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.299801    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb66x\" (UniqueName: \"kubernetes.io/projected/0be46ac4-4bc1-4975-9223-791bd5028769-kube-api-access-kb66x\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.299959 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.299836    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-credential-socket\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.299959 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.299866    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be46ac4-4bc1-4975-9223-791bd5028769-istio-token\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.299959 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.299949    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-istio-data\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.300116 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.299989    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-workload-certs\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.308831 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.308804    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"]
Sep 17 11:11:47.401058 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.401026    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-istio-envoy\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.401058 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.401058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-workload-socket\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.401304 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.401081    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be46ac4-4bc1-4975-9223-791bd5028769-istiod-ca-cert\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.401304 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.401105    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb66x\" (UniqueName: \"kubernetes.io/projected/0be46ac4-4bc1-4975-9223-791bd5028769-kube-api-access-kb66x\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.401304 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.401139    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-credential-socket\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.401304 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.401161    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be46ac4-4bc1-4975-9223-791bd5028769-istio-token\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.401304 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.401195    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-istio-data\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.401304 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.401233    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-workload-certs\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.401979 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.401375    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be46ac4-4bc1-4975-9223-791bd5028769-istio-podinfo\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.401979 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.401471    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-workload-socket\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.401979 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.401580    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-istio-data\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.401979 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.401672    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-workload-certs\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.401979 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.401811    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-credential-socket\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.402188 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.402067    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be46ac4-4bc1-4975-9223-791bd5028769-istiod-ca-cert\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.403595 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.403577    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-istio-envoy\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.403966 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.403950    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be46ac4-4bc1-4975-9223-791bd5028769-istio-podinfo\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.409726 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.409700    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be46ac4-4bc1-4975-9223-791bd5028769-istio-token\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.410434 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.410413    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb66x\" (UniqueName: \"kubernetes.io/projected/0be46ac4-4bc1-4975-9223-791bd5028769-kube-api-access-kb66x\") pod \"e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") " pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.582964 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.582847    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:47.865184 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.865153    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"]
Sep 17 11:11:47.866079 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:11:47.866036    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0be46ac4_4bc1_4975_9223_791bd5028769.slice/crio-31a707be70bf66bd1e269f4b0b717f8de85519faf8ae8f2609c9b9ff9eb341e4 WatchSource:0}: Error finding container 31a707be70bf66bd1e269f4b0b717f8de85519faf8ae8f2609c9b9ff9eb341e4: Status 404 returned error can't find the container with id 31a707be70bf66bd1e269f4b0b717f8de85519faf8ae8f2609c9b9ff9eb341e4
Sep 17 11:11:47.869394 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.869351    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:11:47.869522 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.869442    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:11:47.869522 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.869478    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:11:47.982410 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.982371    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml" event={"ID":"0be46ac4-4bc1-4975-9223-791bd5028769","Type":"ContainerStarted","Data":"b7147623e0d98f1fbbac68c1e62fede13922de9a2d787cf113c736f3113f3e37"}
Sep 17 11:11:47.982603 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:47.982431    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml" event={"ID":"0be46ac4-4bc1-4975-9223-791bd5028769","Type":"ContainerStarted","Data":"31a707be70bf66bd1e269f4b0b717f8de85519faf8ae8f2609c9b9ff9eb341e4"}
Sep 17 11:11:48.003944 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:48.003858    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml" podStartSLOduration=1.003835405 podStartE2EDuration="1.003835405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:11:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:11:48.001616537 +0000 UTC m=+1103.317979694" watchObservedRunningTime="2026-09-17 11:11:48.003835405 +0000 UTC m=+1103.320198561"
Sep 17 11:11:48.583667 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:48.583631    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:48.588943 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:48.588915    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:48.989212 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:48.989177    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:48.990108 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:48.990089    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:11:52.985979 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:52.985942    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"]
Sep 17 11:11:52.988285 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:52.988269    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"
Sep 17 11:11:52.990338 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:52.990317    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-026f86-dockercfg-lgpsw\""
Sep 17 11:11:52.990571 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:52.990550    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-026f86\""
Sep 17 11:11:53.007043 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:53.007017    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"]
Sep 17 11:11:53.058160 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:53.058127    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858\" (UID: \"b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"
Sep 17 11:11:53.058352 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:53.058260    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdkjr\" (UniqueName: \"kubernetes.io/projected/b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da-kube-api-access-vdkjr\") pod \"payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858\" (UID: \"b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"
Sep 17 11:11:53.159286 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:53.159256    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdkjr\" (UniqueName: \"kubernetes.io/projected/b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da-kube-api-access-vdkjr\") pod \"payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858\" (UID: \"b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"
Sep 17 11:11:53.159456 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:53.159348    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858\" (UID: \"b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"
Sep 17 11:11:53.160248 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:53.160225    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858\" (UID: \"b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"
Sep 17 11:11:53.167222 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:53.167196    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdkjr\" (UniqueName: \"kubernetes.io/projected/b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da-kube-api-access-vdkjr\") pod \"payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858\" (UID: \"b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"
Sep 17 11:11:53.299359 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:53.299274    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"
Sep 17 11:11:53.576502 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:53.576479    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"]
Sep 17 11:11:53.578790 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:11:53.578757    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb298ab07_bbbd_4d62_b6ee_8f91ddfdb5da.slice/crio-ac554cdc473a820c11feb2d6c8279bf6ad546d34afd4bc163ea90041e89b6867 WatchSource:0}: Error finding container ac554cdc473a820c11feb2d6c8279bf6ad546d34afd4bc163ea90041e89b6867: Status 404 returned error can't find the container with id ac554cdc473a820c11feb2d6c8279bf6ad546d34afd4bc163ea90041e89b6867
Sep 17 11:11:54.020969 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:54.020929    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858" event={"ID":"b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da","Type":"ContainerStarted","Data":"6b0840db9449c1c2a2794dced168cbd7e9bf921f16d94c23cd9c89e2f0b3fa23"}
Sep 17 11:11:54.021379 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:54.020995    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858" event={"ID":"b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da","Type":"ContainerStarted","Data":"ac554cdc473a820c11feb2d6c8279bf6ad546d34afd4bc163ea90041e89b6867"}
Sep 17 11:11:54.021379 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:54.021026    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"
Sep 17 11:11:54.038199 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:54.038143    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858" podStartSLOduration=2.038123467 podStartE2EDuration="2.038123467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:11:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:11:54.037536841 +0000 UTC m=+1109.353899995" watchObservedRunningTime="2026-09-17 11:11:54.038123467 +0000 UTC m=+1109.354486619"
Sep 17 11:11:56.975921 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:11:56.975866    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"
Sep 17 11:12:05.030218 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:05.030117    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"
Sep 17 11:12:07.187235 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:07.187207    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp"]
Sep 17 11:12:07.190414 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:07.190379    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp"
Sep 17 11:12:07.196592 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:07.196531    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:12:07.196912 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:07.196875    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:12:07.202315 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:07.202019    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp"]
Sep 17 11:12:07.309572 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:07.309527    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73e0cb2-d116-4154-81d1-a84bd5086cd3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp\" (UID: \"b73e0cb2-d116-4154-81d1-a84bd5086cd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp"
Sep 17 11:12:07.410932 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:07.410814    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73e0cb2-d116-4154-81d1-a84bd5086cd3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp\" (UID: \"b73e0cb2-d116-4154-81d1-a84bd5086cd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp"
Sep 17 11:12:07.411793 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:07.411760    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73e0cb2-d116-4154-81d1-a84bd5086cd3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp\" (UID: \"b73e0cb2-d116-4154-81d1-a84bd5086cd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp"
Sep 17 11:12:07.513560 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:07.513529    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp"
Sep 17 11:12:07.844076 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:07.843971    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp"]
Sep 17 11:12:07.856700 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:12:07.856468    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb73e0cb2_d116_4154_81d1_a84bd5086cd3.slice/crio-40a553c3cd87dd598fcc3b1d0fc04552389acb9ac3c8d01dbebd5046e01a9298 WatchSource:0}: Error finding container 40a553c3cd87dd598fcc3b1d0fc04552389acb9ac3c8d01dbebd5046e01a9298: Status 404 returned error can't find the container with id 40a553c3cd87dd598fcc3b1d0fc04552389acb9ac3c8d01dbebd5046e01a9298
Sep 17 11:12:08.118264 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:08.118228    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp" event={"ID":"b73e0cb2-d116-4154-81d1-a84bd5086cd3","Type":"ContainerStarted","Data":"40a553c3cd87dd598fcc3b1d0fc04552389acb9ac3c8d01dbebd5046e01a9298"}
Sep 17 11:12:09.128014 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:09.127928    2564 generic.go:356] "Generic (PLEG): container finished" podID="b73e0cb2-d116-4154-81d1-a84bd5086cd3" containerID="3dbfc0be789e4fc23d7aaa70ba2a1908778101ce0bab1c784fab50de15c77e59" exitCode=0
Sep 17 11:12:09.128450 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:09.128074    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp" event={"ID":"b73e0cb2-d116-4154-81d1-a84bd5086cd3","Type":"ContainerDied","Data":"3dbfc0be789e4fc23d7aaa70ba2a1908778101ce0bab1c784fab50de15c77e59"}
Sep 17 11:12:10.290622 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:10.290596    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp"
Sep 17 11:12:10.444388 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:10.444294    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73e0cb2-d116-4154-81d1-a84bd5086cd3-maas-api-service-ca\") pod \"b73e0cb2-d116-4154-81d1-a84bd5086cd3\" (UID: \"b73e0cb2-d116-4154-81d1-a84bd5086cd3\") "
Sep 17 11:12:10.444694 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:10.444668    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b73e0cb2-d116-4154-81d1-a84bd5086cd3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b73e0cb2-d116-4154-81d1-a84bd5086cd3" (UID: "b73e0cb2-d116-4154-81d1-a84bd5086cd3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:12:10.546030 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:10.545995    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73e0cb2-d116-4154-81d1-a84bd5086cd3-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:12:11.143701 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:11.143675    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp"
Sep 17 11:12:11.146701 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:11.146654    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp" event={"ID":"b73e0cb2-d116-4154-81d1-a84bd5086cd3","Type":"ContainerDied","Data":"40a553c3cd87dd598fcc3b1d0fc04552389acb9ac3c8d01dbebd5046e01a9298"}
Sep 17 11:12:11.146850 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:11.146710    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40a553c3cd87dd598fcc3b1d0fc04552389acb9ac3c8d01dbebd5046e01a9298"
Sep 17 11:12:11.935190 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:11.935145    2564 patch_prober.go:28] interesting pod/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 17 11:12:11.935581 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:11.935228    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn" podUID="95ddf315-2321-4b90-9c13-00f0a603f109" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 17 11:12:12.151922 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:12.151868    2564 generic.go:356] "Generic (PLEG): container finished" podID="95ddf315-2321-4b90-9c13-00f0a603f109" containerID="1bd51abcaad85bda1155cae3ece13f0a28368d23bca096ec71465d650b54c74b" exitCode=0
Sep 17 11:12:12.152107 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:12.151944    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn" event={"ID":"95ddf315-2321-4b90-9c13-00f0a603f109","Type":"ContainerDied","Data":"1bd51abcaad85bda1155cae3ece13f0a28368d23bca096ec71465d650b54c74b"}
Sep 17 11:12:12.152327 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:12.152310    2564 scope.go:117] "RemoveContainer" containerID="1bd51abcaad85bda1155cae3ece13f0a28368d23bca096ec71465d650b54c74b"
Sep 17 11:12:12.153483 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:12.153451    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:12:12.153570 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:12.153529    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:12:12.153619 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:12.153571    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:12:12.153797 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:12.153780    2564 generic.go:356] "Generic (PLEG): container finished" podID="0be46ac4-4bc1-4975-9223-791bd5028769" containerID="b7147623e0d98f1fbbac68c1e62fede13922de9a2d787cf113c736f3113f3e37" exitCode=0
Sep 17 11:12:12.153839 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:12.153831    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml" event={"ID":"0be46ac4-4bc1-4975-9223-791bd5028769","Type":"ContainerDied","Data":"b7147623e0d98f1fbbac68c1e62fede13922de9a2d787cf113c736f3113f3e37"}
Sep 17 11:12:12.154183 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:12.154170    2564 scope.go:117] "RemoveContainer" containerID="b7147623e0d98f1fbbac68c1e62fede13922de9a2d787cf113c736f3113f3e37"
Sep 17 11:12:12.155164 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:12.155136    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:12:12.155257 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:12.155199    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:12:12.155257 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:12.155228    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:12:13.161753 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:13.161715    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn" event={"ID":"95ddf315-2321-4b90-9c13-00f0a603f109","Type":"ContainerStarted","Data":"1cd431630f478153c4407f2f706f03a27463caeb2f2806381e51c79383891ff1"}
Sep 17 11:12:13.165053 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:13.165028    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml" event={"ID":"0be46ac4-4bc1-4975-9223-791bd5028769","Type":"ContainerStarted","Data":"b674cbc7d599f381cd8736fd91bda99f4cf58b1b1c71c76b82285f5da8384f41"}
Sep 17 11:12:13.583865 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:13.583837    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:12:13.584100 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:13.584031    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 17 11:12:13.584209 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:13.584192    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml" podUID="0be46ac4-4bc1-4975-9223-791bd5028769" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 17 11:12:13.969616 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:13.969536    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"]
Sep 17 11:12:13.969800 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:13.969773    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4" podUID="5e58a14e-4e30-4b6e-852d-01ecfb4f3a04" containerName="payload-pre-processing" containerID="cri-o://d243acc7656e85c86cd1ea74998b4365497221a392db8df5253fe570a816fda2" gracePeriod=30
Sep 17 11:12:14.121732 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.121261    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:12:14.121732 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.121390    2564 patch_prober.go:28] interesting pod/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 17 11:12:14.121732 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.121434    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn" podUID="95ddf315-2321-4b90-9c13-00f0a603f109" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 17 11:12:14.174143 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.174111    2564 generic.go:356] "Generic (PLEG): container finished" podID="5e58a14e-4e30-4b6e-852d-01ecfb4f3a04" containerID="d243acc7656e85c86cd1ea74998b4365497221a392db8df5253fe570a816fda2" exitCode=0
Sep 17 11:12:14.174593 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.174172    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4" event={"ID":"5e58a14e-4e30-4b6e-852d-01ecfb4f3a04","Type":"ContainerDied","Data":"d243acc7656e85c86cd1ea74998b4365497221a392db8df5253fe570a816fda2"}
Sep 17 11:12:14.357979 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.357954    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"
Sep 17 11:12:14.448467 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.448369    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp"]
Sep 17 11:12:14.451005 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.450976    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-15f9c946-rh5pp"]
Sep 17 11:12:14.495927 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.492309    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55cs8\" (UniqueName: \"kubernetes.io/projected/5e58a14e-4e30-4b6e-852d-01ecfb4f3a04-kube-api-access-55cs8\") pod \"5e58a14e-4e30-4b6e-852d-01ecfb4f3a04\" (UID: \"5e58a14e-4e30-4b6e-852d-01ecfb4f3a04\") "
Sep 17 11:12:14.495927 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.492373    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e58a14e-4e30-4b6e-852d-01ecfb4f3a04-plugins-config-volume\") pod \"5e58a14e-4e30-4b6e-852d-01ecfb4f3a04\" (UID: \"5e58a14e-4e30-4b6e-852d-01ecfb4f3a04\") "
Sep 17 11:12:14.495927 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.493398    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e58a14e-4e30-4b6e-852d-01ecfb4f3a04-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5e58a14e-4e30-4b6e-852d-01ecfb4f3a04" (UID: "5e58a14e-4e30-4b6e-852d-01ecfb4f3a04"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:12:14.499922 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.496397    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e58a14e-4e30-4b6e-852d-01ecfb4f3a04-kube-api-access-55cs8" (OuterVolumeSpecName: "kube-api-access-55cs8") pod "5e58a14e-4e30-4b6e-852d-01ecfb4f3a04" (UID: "5e58a14e-4e30-4b6e-852d-01ecfb4f3a04"). InnerVolumeSpecName "kube-api-access-55cs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:12:14.564016 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.563978    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"]
Sep 17 11:12:14.584208 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.584175    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 17 11:12:14.584393 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.584231    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml" podUID="0be46ac4-4bc1-4975-9223-791bd5028769" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 17 11:12:14.595975 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.595043    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55cs8\" (UniqueName: \"kubernetes.io/projected/5e58a14e-4e30-4b6e-852d-01ecfb4f3a04-kube-api-access-55cs8\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:12:14.595975 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:14.595075    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e58a14e-4e30-4b6e-852d-01ecfb4f3a04-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:12:15.117807 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:15.117771    2564 patch_prober.go:28] interesting pod/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 17 11:12:15.118155 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:15.118121    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn" podUID="95ddf315-2321-4b90-9c13-00f0a603f109" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 17 11:12:15.147569 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:15.147534    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b73e0cb2-d116-4154-81d1-a84bd5086cd3" path="/var/lib/kubelet/pods/b73e0cb2-d116-4154-81d1-a84bd5086cd3/volumes"
Sep 17 11:12:15.201924 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:15.200360    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"
Sep 17 11:12:15.201924 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:15.200473    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4" event={"ID":"5e58a14e-4e30-4b6e-852d-01ecfb4f3a04","Type":"ContainerDied","Data":"ca8fe0b641d0db3d5d38c9b4bb80d399fc514646bbcba6026d264fa423fa94ac"}
Sep 17 11:12:15.201924 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:15.200553    2564 scope.go:117] "RemoveContainer" containerID="d243acc7656e85c86cd1ea74998b4365497221a392db8df5253fe570a816fda2"
Sep 17 11:12:15.201924 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:15.201002    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn" podUID="95ddf315-2321-4b90-9c13-00f0a603f109" containerName="istio-proxy" containerID="cri-o://1cd431630f478153c4407f2f706f03a27463caeb2f2806381e51c79383891ff1" gracePeriod=30
Sep 17 11:12:15.261325 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:15.261296    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"]
Sep 17 11:12:15.281116 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:15.281091    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-15f9c946-86bd45bbf4-4m6m4"]
Sep 17 11:12:15.583857 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:15.583819    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 17 11:12:15.584061 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:15.583876    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml" podUID="0be46ac4-4bc1-4975-9223-791bd5028769" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 17 11:12:16.584139 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:16.584105    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 17 11:12:16.584514 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:16.584160    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml" podUID="0be46ac4-4bc1-4975-9223-791bd5028769" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 17 11:12:17.146405 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:17.146370    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e58a14e-4e30-4b6e-852d-01ecfb4f3a04" path="/var/lib/kubelet/pods/5e58a14e-4e30-4b6e-852d-01ecfb4f3a04/volumes"
Sep 17 11:12:17.583432 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:17.583392    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 17 11:12:17.583622 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:17.583450    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml" podUID="0be46ac4-4bc1-4975-9223-791bd5028769" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 17 11:12:18.321628 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.321580    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"]
Sep 17 11:12:18.322194 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.322019    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e58a14e-4e30-4b6e-852d-01ecfb4f3a04" containerName="payload-pre-processing"
Sep 17 11:12:18.322194 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.322031    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e58a14e-4e30-4b6e-852d-01ecfb4f3a04" containerName="payload-pre-processing"
Sep 17 11:12:18.322194 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.322041    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b73e0cb2-d116-4154-81d1-a84bd5086cd3" containerName="revoke-keys"
Sep 17 11:12:18.322194 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.322046    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b73e0cb2-d116-4154-81d1-a84bd5086cd3" containerName="revoke-keys"
Sep 17 11:12:18.322194 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.322131    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b73e0cb2-d116-4154-81d1-a84bd5086cd3" containerName="revoke-keys"
Sep 17 11:12:18.322194 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.322143    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e58a14e-4e30-4b6e-852d-01ecfb4f3a04" containerName="payload-pre-processing"
Sep 17 11:12:18.325464 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.325439    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"
Sep 17 11:12:18.328267 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.328238    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-b8287f\""
Sep 17 11:12:18.328428 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.328274    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-b8287f-dockercfg-vgqkq\""
Sep 17 11:12:18.351018 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.350984    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"]
Sep 17 11:12:18.446781 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.446743    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwtsv\" (UniqueName: \"kubernetes.io/projected/66546020-c5a6-4761-8b6a-2f4cbb4265ad-kube-api-access-qwtsv\") pod \"payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m\" (UID: \"66546020-c5a6-4761-8b6a-2f4cbb4265ad\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"
Sep 17 11:12:18.446961 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.446802    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66546020-c5a6-4761-8b6a-2f4cbb4265ad-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m\" (UID: \"66546020-c5a6-4761-8b6a-2f4cbb4265ad\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"
Sep 17 11:12:18.547375 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.547340    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwtsv\" (UniqueName: \"kubernetes.io/projected/66546020-c5a6-4761-8b6a-2f4cbb4265ad-kube-api-access-qwtsv\") pod \"payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m\" (UID: \"66546020-c5a6-4761-8b6a-2f4cbb4265ad\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"
Sep 17 11:12:18.547375 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.547382    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66546020-c5a6-4761-8b6a-2f4cbb4265ad-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m\" (UID: \"66546020-c5a6-4761-8b6a-2f4cbb4265ad\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"
Sep 17 11:12:18.548160 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.548137    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66546020-c5a6-4761-8b6a-2f4cbb4265ad-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m\" (UID: \"66546020-c5a6-4761-8b6a-2f4cbb4265ad\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"
Sep 17 11:12:18.558487 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.558458    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwtsv\" (UniqueName: \"kubernetes.io/projected/66546020-c5a6-4761-8b6a-2f4cbb4265ad-kube-api-access-qwtsv\") pod \"payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m\" (UID: \"66546020-c5a6-4761-8b6a-2f4cbb4265ad\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"
Sep 17 11:12:18.588088 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.588012    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:12:18.588305 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.588280    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:12:18.589073 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.589053    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:12:18.636476 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.636437    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"
Sep 17 11:12:18.929819 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:18.929793    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"]
Sep 17 11:12:18.931682 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:12:18.931655    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66546020_c5a6_4761_8b6a_2f4cbb4265ad.slice/crio-e952e985b3381b9d9ccb6ab8d7f5d04e184b858d041b156596c7a4aec19fbd89 WatchSource:0}: Error finding container e952e985b3381b9d9ccb6ab8d7f5d04e184b858d041b156596c7a4aec19fbd89: Status 404 returned error can't find the container with id e952e985b3381b9d9ccb6ab8d7f5d04e184b858d041b156596c7a4aec19fbd89
Sep 17 11:12:19.137441 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:19.137410    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:12:19.245444 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:19.245410    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m" event={"ID":"66546020-c5a6-4761-8b6a-2f4cbb4265ad","Type":"ContainerStarted","Data":"28c6bb27d85dd04f8533cb4989dc369934e85e4121a07130a7ea68b1cdf89884"}
Sep 17 11:12:19.245695 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:19.245480    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m" event={"ID":"66546020-c5a6-4761-8b6a-2f4cbb4265ad","Type":"ContainerStarted","Data":"e952e985b3381b9d9ccb6ab8d7f5d04e184b858d041b156596c7a4aec19fbd89"}
Sep 17 11:12:19.245814 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:19.245794    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"
Sep 17 11:12:23.860184 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:23.860163    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:12:23.885809 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:23.885761    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m" podStartSLOduration=5.885742956 podStartE2EDuration="5.885742956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:12:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:12:19.267980567 +0000 UTC m=+1134.584343724" watchObservedRunningTime="2026-09-17 11:12:23.885742956 +0000 UTC m=+1139.202106112"
Sep 17 11:12:24.003633 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.003608    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-workload-certs\") pod \"95ddf315-2321-4b90-9c13-00f0a603f109\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") "
Sep 17 11:12:24.003798 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.003646    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95ddf315-2321-4b90-9c13-00f0a603f109-istio-podinfo\") pod \"95ddf315-2321-4b90-9c13-00f0a603f109\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") "
Sep 17 11:12:24.003798 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.003680    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95ddf315-2321-4b90-9c13-00f0a603f109-istiod-ca-cert\") pod \"95ddf315-2321-4b90-9c13-00f0a603f109\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") "
Sep 17 11:12:24.003798 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.003712    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95ddf315-2321-4b90-9c13-00f0a603f109-istio-token\") pod \"95ddf315-2321-4b90-9c13-00f0a603f109\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") "
Sep 17 11:12:24.003798 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.003736    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-istio-envoy\") pod \"95ddf315-2321-4b90-9c13-00f0a603f109\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") "
Sep 17 11:12:24.003798 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.003758    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-workload-socket\") pod \"95ddf315-2321-4b90-9c13-00f0a603f109\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") "
Sep 17 11:12:24.003798 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.003782    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-istio-data\") pod \"95ddf315-2321-4b90-9c13-00f0a603f109\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") "
Sep 17 11:12:24.004151 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.003831    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-credential-socket\") pod \"95ddf315-2321-4b90-9c13-00f0a603f109\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") "
Sep 17 11:12:24.004151 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.003880    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbfh2\" (UniqueName: \"kubernetes.io/projected/95ddf315-2321-4b90-9c13-00f0a603f109-kube-api-access-bbfh2\") pod \"95ddf315-2321-4b90-9c13-00f0a603f109\" (UID: \"95ddf315-2321-4b90-9c13-00f0a603f109\") "
Sep 17 11:12:24.004151 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.003964    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "95ddf315-2321-4b90-9c13-00f0a603f109" (UID: "95ddf315-2321-4b90-9c13-00f0a603f109"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:12:24.004151 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.004086    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "95ddf315-2321-4b90-9c13-00f0a603f109" (UID: "95ddf315-2321-4b90-9c13-00f0a603f109"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:12:24.004360 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.004237    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-istio-data" (OuterVolumeSpecName: "istio-data") pod "95ddf315-2321-4b90-9c13-00f0a603f109" (UID: "95ddf315-2321-4b90-9c13-00f0a603f109"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:12:24.004360 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.004248    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "95ddf315-2321-4b90-9c13-00f0a603f109" (UID: "95ddf315-2321-4b90-9c13-00f0a603f109"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:12:24.004360 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.004271    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95ddf315-2321-4b90-9c13-00f0a603f109-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "95ddf315-2321-4b90-9c13-00f0a603f109" (UID: "95ddf315-2321-4b90-9c13-00f0a603f109"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:12:24.004360 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.004306    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-workload-certs\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:12:24.004360 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.004325    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-workload-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:12:24.006050 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.006026    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/95ddf315-2321-4b90-9c13-00f0a603f109-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "95ddf315-2321-4b90-9c13-00f0a603f109" (UID: "95ddf315-2321-4b90-9c13-00f0a603f109"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 11:12:24.006050 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.006039    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95ddf315-2321-4b90-9c13-00f0a603f109-istio-token" (OuterVolumeSpecName: "istio-token") pod "95ddf315-2321-4b90-9c13-00f0a603f109" (UID: "95ddf315-2321-4b90-9c13-00f0a603f109"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:12:24.006603 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.006572    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "95ddf315-2321-4b90-9c13-00f0a603f109" (UID: "95ddf315-2321-4b90-9c13-00f0a603f109"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:12:24.006687 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.006609    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95ddf315-2321-4b90-9c13-00f0a603f109-kube-api-access-bbfh2" (OuterVolumeSpecName: "kube-api-access-bbfh2") pod "95ddf315-2321-4b90-9c13-00f0a603f109" (UID: "95ddf315-2321-4b90-9c13-00f0a603f109"). InnerVolumeSpecName "kube-api-access-bbfh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:12:24.105321 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.105292    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-istio-data\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:12:24.105321 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.105317    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-credential-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:12:24.105321 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.105327    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbfh2\" (UniqueName: \"kubernetes.io/projected/95ddf315-2321-4b90-9c13-00f0a603f109-kube-api-access-bbfh2\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:12:24.105516 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.105336    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95ddf315-2321-4b90-9c13-00f0a603f109-istio-podinfo\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:12:24.105516 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.105345    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95ddf315-2321-4b90-9c13-00f0a603f109-istiod-ca-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:12:24.105516 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.105353    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95ddf315-2321-4b90-9c13-00f0a603f109-istio-token\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:12:24.105516 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.105360    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95ddf315-2321-4b90-9c13-00f0a603f109-istio-envoy\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:12:24.278453 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.278367    2564 generic.go:356] "Generic (PLEG): container finished" podID="95ddf315-2321-4b90-9c13-00f0a603f109" containerID="1cd431630f478153c4407f2f706f03a27463caeb2f2806381e51c79383891ff1" exitCode=0
Sep 17 11:12:24.278453 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.278421    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"
Sep 17 11:12:24.278671 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.278447    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn" event={"ID":"95ddf315-2321-4b90-9c13-00f0a603f109","Type":"ContainerDied","Data":"1cd431630f478153c4407f2f706f03a27463caeb2f2806381e51c79383891ff1"}
Sep 17 11:12:24.278671 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.278499    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn" event={"ID":"95ddf315-2321-4b90-9c13-00f0a603f109","Type":"ContainerDied","Data":"f652756a58372df437728a3fe193cc4d2c2678441b66831b478f283edb6a81f8"}
Sep 17 11:12:24.278671 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.278521    2564 scope.go:117] "RemoveContainer" containerID="1cd431630f478153c4407f2f706f03a27463caeb2f2806381e51c79383891ff1"
Sep 17 11:12:24.295874 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.295857    2564 scope.go:117] "RemoveContainer" containerID="1bd51abcaad85bda1155cae3ece13f0a28368d23bca096ec71465d650b54c74b"
Sep 17 11:12:24.305547 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.305350    2564 scope.go:117] "RemoveContainer" containerID="1cd431630f478153c4407f2f706f03a27463caeb2f2806381e51c79383891ff1"
Sep 17 11:12:24.305737 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:12:24.305709    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cd431630f478153c4407f2f706f03a27463caeb2f2806381e51c79383891ff1\": container with ID starting with 1cd431630f478153c4407f2f706f03a27463caeb2f2806381e51c79383891ff1 not found: ID does not exist" containerID="1cd431630f478153c4407f2f706f03a27463caeb2f2806381e51c79383891ff1"
Sep 17 11:12:24.305814 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.305747    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cd431630f478153c4407f2f706f03a27463caeb2f2806381e51c79383891ff1"} err="failed to get container status \"1cd431630f478153c4407f2f706f03a27463caeb2f2806381e51c79383891ff1\": rpc error: code = NotFound desc = could not find container \"1cd431630f478153c4407f2f706f03a27463caeb2f2806381e51c79383891ff1\": container with ID starting with 1cd431630f478153c4407f2f706f03a27463caeb2f2806381e51c79383891ff1 not found: ID does not exist"
Sep 17 11:12:24.305814 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.305773    2564 scope.go:117] "RemoveContainer" containerID="1bd51abcaad85bda1155cae3ece13f0a28368d23bca096ec71465d650b54c74b"
Sep 17 11:12:24.306103 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:12:24.306080    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bd51abcaad85bda1155cae3ece13f0a28368d23bca096ec71465d650b54c74b\": container with ID starting with 1bd51abcaad85bda1155cae3ece13f0a28368d23bca096ec71465d650b54c74b not found: ID does not exist" containerID="1bd51abcaad85bda1155cae3ece13f0a28368d23bca096ec71465d650b54c74b"
Sep 17 11:12:24.306199 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.306110    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bd51abcaad85bda1155cae3ece13f0a28368d23bca096ec71465d650b54c74b"} err="failed to get container status \"1bd51abcaad85bda1155cae3ece13f0a28368d23bca096ec71465d650b54c74b\": rpc error: code = NotFound desc = could not find container \"1bd51abcaad85bda1155cae3ece13f0a28368d23bca096ec71465d650b54c74b\": container with ID starting with 1bd51abcaad85bda1155cae3ece13f0a28368d23bca096ec71465d650b54c74b not found: ID does not exist"
Sep 17 11:12:24.307449 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.307426    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"]
Sep 17 11:12:24.312379 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:24.312353    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-15f9c946-openshift-default-5ddcd5565b-tvhxn"]
Sep 17 11:12:25.145484 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:25.145452    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95ddf315-2321-4b90-9c13-00f0a603f109" path="/var/lib/kubelet/pods/95ddf315-2321-4b90-9c13-00f0a603f109/volumes"
Sep 17 11:12:30.254458 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:30.254426    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"
Sep 17 11:12:46.137685 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:12:46.137654    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:13:12.061488 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:12.061456    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6"]
Sep 17 11:13:12.061936 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:12.061883    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95ddf315-2321-4b90-9c13-00f0a603f109" containerName="istio-proxy"
Sep 17 11:13:12.061936 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:12.061920    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="95ddf315-2321-4b90-9c13-00f0a603f109" containerName="istio-proxy"
Sep 17 11:13:12.061936 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:12.061929    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95ddf315-2321-4b90-9c13-00f0a603f109" containerName="istio-proxy"
Sep 17 11:13:12.061936 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:12.061934    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="95ddf315-2321-4b90-9c13-00f0a603f109" containerName="istio-proxy"
Sep 17 11:13:12.062084 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:12.062015    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="95ddf315-2321-4b90-9c13-00f0a603f109" containerName="istio-proxy"
Sep 17 11:13:12.062084 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:12.062023    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="95ddf315-2321-4b90-9c13-00f0a603f109" containerName="istio-proxy"
Sep 17 11:13:12.064108 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:12.064093    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6"
Sep 17 11:13:12.065980 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:12.065962    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:13:12.066061 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:12.066005    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:13:12.072222 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:12.072069    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6"]
Sep 17 11:13:12.242079 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:12.242047    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0557f635-7f09-40d7-9f65-30d31bec4730-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6\" (UID: \"0557f635-7f09-40d7-9f65-30d31bec4730\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6"
Sep 17 11:13:12.342779 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:12.342709    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0557f635-7f09-40d7-9f65-30d31bec4730-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6\" (UID: \"0557f635-7f09-40d7-9f65-30d31bec4730\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6"
Sep 17 11:13:12.343314 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:12.343296    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0557f635-7f09-40d7-9f65-30d31bec4730-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6\" (UID: \"0557f635-7f09-40d7-9f65-30d31bec4730\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6"
Sep 17 11:13:12.376276 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:12.376254    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6"
Sep 17 11:13:12.649449 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:12.649423    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6"]
Sep 17 11:13:12.651536 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:13:12.651508    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0557f635_7f09_40d7_9f65_30d31bec4730.slice/crio-3d7bfc326d505966ec97e71bf3fcd2fda6904e33a9c8d580b77363c9dc835165 WatchSource:0}: Error finding container 3d7bfc326d505966ec97e71bf3fcd2fda6904e33a9c8d580b77363c9dc835165: Status 404 returned error can't find the container with id 3d7bfc326d505966ec97e71bf3fcd2fda6904e33a9c8d580b77363c9dc835165
Sep 17 11:13:13.574156 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:13.574123    2564 generic.go:356] "Generic (PLEG): container finished" podID="0557f635-7f09-40d7-9f65-30d31bec4730" containerID="0494857c37cf526d091bb9902cb9dcc9abd4eb1cfc94c294a482dc76373f6e60" exitCode=0
Sep 17 11:13:13.574514 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:13.574209    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6" event={"ID":"0557f635-7f09-40d7-9f65-30d31bec4730","Type":"ContainerDied","Data":"0494857c37cf526d091bb9902cb9dcc9abd4eb1cfc94c294a482dc76373f6e60"}
Sep 17 11:13:13.574514 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:13.574260    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6" event={"ID":"0557f635-7f09-40d7-9f65-30d31bec4730","Type":"ContainerStarted","Data":"3d7bfc326d505966ec97e71bf3fcd2fda6904e33a9c8d580b77363c9dc835165"}
Sep 17 11:13:14.719527 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:14.719503    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6"
Sep 17 11:13:14.869541 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:14.869463    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0557f635-7f09-40d7-9f65-30d31bec4730-maas-api-service-ca\") pod \"0557f635-7f09-40d7-9f65-30d31bec4730\" (UID: \"0557f635-7f09-40d7-9f65-30d31bec4730\") "
Sep 17 11:13:14.869803 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:14.869782    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0557f635-7f09-40d7-9f65-30d31bec4730-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0557f635-7f09-40d7-9f65-30d31bec4730" (UID: "0557f635-7f09-40d7-9f65-30d31bec4730"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:13:14.970578 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:14.970545    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0557f635-7f09-40d7-9f65-30d31bec4730-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:15.587830 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:15.587801    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6"
Sep 17 11:13:15.588035 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:15.587845    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6" event={"ID":"0557f635-7f09-40d7-9f65-30d31bec4730","Type":"ContainerDied","Data":"3d7bfc326d505966ec97e71bf3fcd2fda6904e33a9c8d580b77363c9dc835165"}
Sep 17 11:13:15.588035 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:15.587924    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d7bfc326d505966ec97e71bf3fcd2fda6904e33a9c8d580b77363c9dc835165"
Sep 17 11:13:16.948283 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:16.948247    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"]
Sep 17 11:13:27.705076 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:27.705035    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"]
Sep 17 11:13:27.705513 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:27.705254    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p" podUID="8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe" containerName="payload-processing" containerID="cri-o://5af5213b8d8e2608a89c3583e11d5400a47f7ce2c1852f88ff1d6b254d5814fe" gracePeriod=30
Sep 17 11:13:27.714685 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:27.714656    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"]
Sep 17 11:13:27.714956 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:27.714891    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c" podUID="b3730c4b-e89d-41e6-afb5-07a4465f218b" containerName="payload-pre-processing" containerID="cri-o://8f915154046d4a522fffa6838ed8556be853a1176e6a6e0d48ac98613af402c0" gracePeriod=30
Sep 17 11:13:27.957439 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:27.957354    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6"]
Sep 17 11:13:27.960858 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:27.960825    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-766cc3-m9zq6"]
Sep 17 11:13:28.007533 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.007511    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"
Sep 17 11:13:28.010660 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.010643    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"
Sep 17 11:13:28.090273 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.090244    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9676\" (UniqueName: \"kubernetes.io/projected/b3730c4b-e89d-41e6-afb5-07a4465f218b-kube-api-access-n9676\") pod \"b3730c4b-e89d-41e6-afb5-07a4465f218b\" (UID: \"b3730c4b-e89d-41e6-afb5-07a4465f218b\") "
Sep 17 11:13:28.090461 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.090284    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vj2t\" (UniqueName: \"kubernetes.io/projected/8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe-kube-api-access-6vj2t\") pod \"8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe\" (UID: \"8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe\") "
Sep 17 11:13:28.090461 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.090337    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe-plugins-config-volume\") pod \"8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe\" (UID: \"8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe\") "
Sep 17 11:13:28.090461 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.090412    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3730c4b-e89d-41e6-afb5-07a4465f218b-plugins-config-volume\") pod \"b3730c4b-e89d-41e6-afb5-07a4465f218b\" (UID: \"b3730c4b-e89d-41e6-afb5-07a4465f218b\") "
Sep 17 11:13:28.090750 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.090722    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3730c4b-e89d-41e6-afb5-07a4465f218b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b3730c4b-e89d-41e6-afb5-07a4465f218b" (UID: "b3730c4b-e89d-41e6-afb5-07a4465f218b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:13:28.090808 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.090746    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe" (UID: "8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:13:28.092495 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.092469    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3730c4b-e89d-41e6-afb5-07a4465f218b-kube-api-access-n9676" (OuterVolumeSpecName: "kube-api-access-n9676") pod "b3730c4b-e89d-41e6-afb5-07a4465f218b" (UID: "b3730c4b-e89d-41e6-afb5-07a4465f218b"). InnerVolumeSpecName "kube-api-access-n9676". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:13:28.092584 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.092538    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe-kube-api-access-6vj2t" (OuterVolumeSpecName: "kube-api-access-6vj2t") pod "8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe" (UID: "8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe"). InnerVolumeSpecName "kube-api-access-6vj2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:13:28.191491 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.191462    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:28.191491 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.191488    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3730c4b-e89d-41e6-afb5-07a4465f218b-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:28.191658 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.191497    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9676\" (UniqueName: \"kubernetes.io/projected/b3730c4b-e89d-41e6-afb5-07a4465f218b-kube-api-access-n9676\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:28.191658 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.191507    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vj2t\" (UniqueName: \"kubernetes.io/projected/8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe-kube-api-access-6vj2t\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:28.674142 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.674110    2564 generic.go:356] "Generic (PLEG): container finished" podID="8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe" containerID="5af5213b8d8e2608a89c3583e11d5400a47f7ce2c1852f88ff1d6b254d5814fe" exitCode=0
Sep 17 11:13:28.674305 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.674155    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"
Sep 17 11:13:28.674305 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.674196    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p" event={"ID":"8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe","Type":"ContainerDied","Data":"5af5213b8d8e2608a89c3583e11d5400a47f7ce2c1852f88ff1d6b254d5814fe"}
Sep 17 11:13:28.674305 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.674250    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p" event={"ID":"8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe","Type":"ContainerDied","Data":"64dbd3eab9f99b2460c97c79d8a069dd39d8889f9992805d6ccdd5c0d5c2a32d"}
Sep 17 11:13:28.674305 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.674280    2564 scope.go:117] "RemoveContainer" containerID="5af5213b8d8e2608a89c3583e11d5400a47f7ce2c1852f88ff1d6b254d5814fe"
Sep 17 11:13:28.675592 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.675566    2564 generic.go:356] "Generic (PLEG): container finished" podID="b3730c4b-e89d-41e6-afb5-07a4465f218b" containerID="8f915154046d4a522fffa6838ed8556be853a1176e6a6e0d48ac98613af402c0" exitCode=0
Sep 17 11:13:28.675721 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.675601    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"
Sep 17 11:13:28.675721 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.675653    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c" event={"ID":"b3730c4b-e89d-41e6-afb5-07a4465f218b","Type":"ContainerDied","Data":"8f915154046d4a522fffa6838ed8556be853a1176e6a6e0d48ac98613af402c0"}
Sep 17 11:13:28.675721 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.675700    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c" event={"ID":"b3730c4b-e89d-41e6-afb5-07a4465f218b","Type":"ContainerDied","Data":"6539ec11e855c299b5f42719c9ccdf264f50319652dab7bcd8747297b29a9bee"}
Sep 17 11:13:28.684732 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.684596    2564 scope.go:117] "RemoveContainer" containerID="5af5213b8d8e2608a89c3583e11d5400a47f7ce2c1852f88ff1d6b254d5814fe"
Sep 17 11:13:28.684962 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:13:28.684913    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5af5213b8d8e2608a89c3583e11d5400a47f7ce2c1852f88ff1d6b254d5814fe\": container with ID starting with 5af5213b8d8e2608a89c3583e11d5400a47f7ce2c1852f88ff1d6b254d5814fe not found: ID does not exist" containerID="5af5213b8d8e2608a89c3583e11d5400a47f7ce2c1852f88ff1d6b254d5814fe"
Sep 17 11:13:28.684962 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.684947    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5af5213b8d8e2608a89c3583e11d5400a47f7ce2c1852f88ff1d6b254d5814fe"} err="failed to get container status \"5af5213b8d8e2608a89c3583e11d5400a47f7ce2c1852f88ff1d6b254d5814fe\": rpc error: code = NotFound desc = could not find container \"5af5213b8d8e2608a89c3583e11d5400a47f7ce2c1852f88ff1d6b254d5814fe\": container with ID starting with 5af5213b8d8e2608a89c3583e11d5400a47f7ce2c1852f88ff1d6b254d5814fe not found: ID does not exist"
Sep 17 11:13:28.685106 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.684969    2564 scope.go:117] "RemoveContainer" containerID="8f915154046d4a522fffa6838ed8556be853a1176e6a6e0d48ac98613af402c0"
Sep 17 11:13:28.693619 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.693604    2564 scope.go:117] "RemoveContainer" containerID="8f915154046d4a522fffa6838ed8556be853a1176e6a6e0d48ac98613af402c0"
Sep 17 11:13:28.693862 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:13:28.693845    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f915154046d4a522fffa6838ed8556be853a1176e6a6e0d48ac98613af402c0\": container with ID starting with 8f915154046d4a522fffa6838ed8556be853a1176e6a6e0d48ac98613af402c0 not found: ID does not exist" containerID="8f915154046d4a522fffa6838ed8556be853a1176e6a6e0d48ac98613af402c0"
Sep 17 11:13:28.693961 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.693869    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f915154046d4a522fffa6838ed8556be853a1176e6a6e0d48ac98613af402c0"} err="failed to get container status \"8f915154046d4a522fffa6838ed8556be853a1176e6a6e0d48ac98613af402c0\": rpc error: code = NotFound desc = could not find container \"8f915154046d4a522fffa6838ed8556be853a1176e6a6e0d48ac98613af402c0\": container with ID starting with 8f915154046d4a522fffa6838ed8556be853a1176e6a6e0d48ac98613af402c0 not found: ID does not exist"
Sep 17 11:13:28.698368 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.698349    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"]
Sep 17 11:13:28.702567 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.702548    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-766cc3-859f9c8f8b-rnj8c"]
Sep 17 11:13:28.711817 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.711793    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"]
Sep 17 11:13:28.719389 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:28.719368    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-766cc3-7949cbb557-wk99p"]
Sep 17 11:13:29.143654 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:29.143617    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0557f635-7f09-40d7-9f65-30d31bec4730" path="/var/lib/kubelet/pods/0557f635-7f09-40d7-9f65-30d31bec4730/volumes"
Sep 17 11:13:29.144022 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:29.144008    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe" path="/var/lib/kubelet/pods/8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe/volumes"
Sep 17 11:13:29.144350 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:29.144338    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3730c4b-e89d-41e6-afb5-07a4465f218b" path="/var/lib/kubelet/pods/b3730c4b-e89d-41e6-afb5-07a4465f218b/volumes"
Sep 17 11:13:31.968700 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:31.968652    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs" podUID="e0365f11-a09b-4a19-be32-0323f7d0bd0e" containerName="main" containerID="cri-o://9f6814ae764d11feb28a253b8b3ab5e1ac6358ed2871a235263deba6d38700ef" gracePeriod=45
Sep 17 11:13:32.227368 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.227312    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:13:32.328464 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.328437    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0365f11-a09b-4a19-be32-0323f7d0bd0e-tls-certs\") pod \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") "
Sep 17 11:13:32.328652 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.328502    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-tmp-dir\") pod \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") "
Sep 17 11:13:32.328652 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.328586    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46prx\" (UniqueName: \"kubernetes.io/projected/e0365f11-a09b-4a19-be32-0323f7d0bd0e-kube-api-access-46prx\") pod \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") "
Sep 17 11:13:32.328652 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.328636    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-dshm\") pod \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") "
Sep 17 11:13:32.328834 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.328695    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-model-cache\") pod \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") "
Sep 17 11:13:32.328834 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.328721    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-home\") pod \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\" (UID: \"e0365f11-a09b-4a19-be32-0323f7d0bd0e\") "
Sep 17 11:13:32.328968 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.328827    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e0365f11-a09b-4a19-be32-0323f7d0bd0e" (UID: "e0365f11-a09b-4a19-be32-0323f7d0bd0e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:13:32.329057 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.329032    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-model-cache" (OuterVolumeSpecName: "model-cache") pod "e0365f11-a09b-4a19-be32-0323f7d0bd0e" (UID: "e0365f11-a09b-4a19-be32-0323f7d0bd0e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:13:32.329106 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.329064    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-home" (OuterVolumeSpecName: "home") pod "e0365f11-a09b-4a19-be32-0323f7d0bd0e" (UID: "e0365f11-a09b-4a19-be32-0323f7d0bd0e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:13:32.329251 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.329175    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-model-cache\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:32.329322 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.329311    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-home\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:32.329380 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.329328    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-tmp-dir\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:32.330837 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.330806    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0365f11-a09b-4a19-be32-0323f7d0bd0e-kube-api-access-46prx" (OuterVolumeSpecName: "kube-api-access-46prx") pod "e0365f11-a09b-4a19-be32-0323f7d0bd0e" (UID: "e0365f11-a09b-4a19-be32-0323f7d0bd0e"). InnerVolumeSpecName "kube-api-access-46prx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:13:32.330837 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.330820    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0365f11-a09b-4a19-be32-0323f7d0bd0e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e0365f11-a09b-4a19-be32-0323f7d0bd0e" (UID: "e0365f11-a09b-4a19-be32-0323f7d0bd0e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 11:13:32.331209 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.331188    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-dshm" (OuterVolumeSpecName: "dshm") pod "e0365f11-a09b-4a19-be32-0323f7d0bd0e" (UID: "e0365f11-a09b-4a19-be32-0323f7d0bd0e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:13:32.430791 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.430762    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46prx\" (UniqueName: \"kubernetes.io/projected/e0365f11-a09b-4a19-be32-0323f7d0bd0e-kube-api-access-46prx\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:32.430791 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.430785    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0365f11-a09b-4a19-be32-0323f7d0bd0e-dshm\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:32.430791 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.430794    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0365f11-a09b-4a19-be32-0323f7d0bd0e-tls-certs\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:32.701456 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.701424    2564 generic.go:356] "Generic (PLEG): container finished" podID="e0365f11-a09b-4a19-be32-0323f7d0bd0e" containerID="9f6814ae764d11feb28a253b8b3ab5e1ac6358ed2871a235263deba6d38700ef" exitCode=0
Sep 17 11:13:32.701620 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.701468    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"
Sep 17 11:13:32.701620 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.701499    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs" event={"ID":"e0365f11-a09b-4a19-be32-0323f7d0bd0e","Type":"ContainerDied","Data":"9f6814ae764d11feb28a253b8b3ab5e1ac6358ed2871a235263deba6d38700ef"}
Sep 17 11:13:32.701620 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.701538    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs" event={"ID":"e0365f11-a09b-4a19-be32-0323f7d0bd0e","Type":"ContainerDied","Data":"afacc939ab50630e871effa0221c3f3e38f6f87a1e8f8996a48c962b5d4b35d5"}
Sep 17 11:13:32.701620 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.701555    2564 scope.go:117] "RemoveContainer" containerID="9f6814ae764d11feb28a253b8b3ab5e1ac6358ed2871a235263deba6d38700ef"
Sep 17 11:13:32.715110 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.715096    2564 scope.go:117] "RemoveContainer" containerID="9f6814ae764d11feb28a253b8b3ab5e1ac6358ed2871a235263deba6d38700ef"
Sep 17 11:13:32.715360 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:13:32.715344    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f6814ae764d11feb28a253b8b3ab5e1ac6358ed2871a235263deba6d38700ef\": container with ID starting with 9f6814ae764d11feb28a253b8b3ab5e1ac6358ed2871a235263deba6d38700ef not found: ID does not exist" containerID="9f6814ae764d11feb28a253b8b3ab5e1ac6358ed2871a235263deba6d38700ef"
Sep 17 11:13:32.715399 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.715370    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f6814ae764d11feb28a253b8b3ab5e1ac6358ed2871a235263deba6d38700ef"} err="failed to get container status \"9f6814ae764d11feb28a253b8b3ab5e1ac6358ed2871a235263deba6d38700ef\": rpc error: code = NotFound desc = could not find container \"9f6814ae764d11feb28a253b8b3ab5e1ac6358ed2871a235263deba6d38700ef\": container with ID starting with 9f6814ae764d11feb28a253b8b3ab5e1ac6358ed2871a235263deba6d38700ef not found: ID does not exist"
Sep 17 11:13:32.724373 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.724346    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"]
Sep 17 11:13:32.731935 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:32.731915    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-766cc3/premium-w3-766cc3-kserve-774bfb45bf-spjzs"]
Sep 17 11:13:33.142948 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:33.142891    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0365f11-a09b-4a19-be32-0323f7d0bd0e" path="/var/lib/kubelet/pods/e0365f11-a09b-4a19-be32-0323f7d0bd0e/volumes"
Sep 17 11:13:34.447076 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:34.447039    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"]
Sep 17 11:13:34.447538 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:34.447261    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv" podUID="bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" containerName="istio-proxy" containerID="cri-o://8c11eee5f8056c0458f453bdb19300d118ab1d156386348b099ea8415f112718" gracePeriod=30
Sep 17 11:13:38.582541 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:38.582499    2564 patch_prober.go:28] interesting pod/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 17 11:13:38.583002 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:38.582566    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv" podUID="bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 11:13:39.695194 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.695169    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:13:39.745939 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.745862    2564 generic.go:356] "Generic (PLEG): container finished" podID="bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" containerID="8c11eee5f8056c0458f453bdb19300d118ab1d156386348b099ea8415f112718" exitCode=0
Sep 17 11:13:39.745939 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.745913    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"
Sep 17 11:13:39.746135 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.745942    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv" event={"ID":"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac","Type":"ContainerDied","Data":"8c11eee5f8056c0458f453bdb19300d118ab1d156386348b099ea8415f112718"}
Sep 17 11:13:39.746135 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.745998    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv" event={"ID":"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac","Type":"ContainerDied","Data":"bb30f6f5e84654076f145c5806537103b55c94f9e90ff7b714cbb9aa2a6601c7"}
Sep 17 11:13:39.746135 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.746030    2564 scope.go:117] "RemoveContainer" containerID="8c11eee5f8056c0458f453bdb19300d118ab1d156386348b099ea8415f112718"
Sep 17 11:13:39.755461 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.755442    2564 scope.go:117] "RemoveContainer" containerID="7f2e9c6b9f928f45d81edc3c5a512bfbf0b1d1102be75a97a1bc487b0d2e51a7"
Sep 17 11:13:39.763706 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.763688    2564 scope.go:117] "RemoveContainer" containerID="8c11eee5f8056c0458f453bdb19300d118ab1d156386348b099ea8415f112718"
Sep 17 11:13:39.763995 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:13:39.763976    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c11eee5f8056c0458f453bdb19300d118ab1d156386348b099ea8415f112718\": container with ID starting with 8c11eee5f8056c0458f453bdb19300d118ab1d156386348b099ea8415f112718 not found: ID does not exist" containerID="8c11eee5f8056c0458f453bdb19300d118ab1d156386348b099ea8415f112718"
Sep 17 11:13:39.764084 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.764008    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c11eee5f8056c0458f453bdb19300d118ab1d156386348b099ea8415f112718"} err="failed to get container status \"8c11eee5f8056c0458f453bdb19300d118ab1d156386348b099ea8415f112718\": rpc error: code = NotFound desc = could not find container \"8c11eee5f8056c0458f453bdb19300d118ab1d156386348b099ea8415f112718\": container with ID starting with 8c11eee5f8056c0458f453bdb19300d118ab1d156386348b099ea8415f112718 not found: ID does not exist"
Sep 17 11:13:39.764084 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.764030    2564 scope.go:117] "RemoveContainer" containerID="7f2e9c6b9f928f45d81edc3c5a512bfbf0b1d1102be75a97a1bc487b0d2e51a7"
Sep 17 11:13:39.764291 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:13:39.764274    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f2e9c6b9f928f45d81edc3c5a512bfbf0b1d1102be75a97a1bc487b0d2e51a7\": container with ID starting with 7f2e9c6b9f928f45d81edc3c5a512bfbf0b1d1102be75a97a1bc487b0d2e51a7 not found: ID does not exist" containerID="7f2e9c6b9f928f45d81edc3c5a512bfbf0b1d1102be75a97a1bc487b0d2e51a7"
Sep 17 11:13:39.764346 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.764299    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f2e9c6b9f928f45d81edc3c5a512bfbf0b1d1102be75a97a1bc487b0d2e51a7"} err="failed to get container status \"7f2e9c6b9f928f45d81edc3c5a512bfbf0b1d1102be75a97a1bc487b0d2e51a7\": rpc error: code = NotFound desc = could not find container \"7f2e9c6b9f928f45d81edc3c5a512bfbf0b1d1102be75a97a1bc487b0d2e51a7\": container with ID starting with 7f2e9c6b9f928f45d81edc3c5a512bfbf0b1d1102be75a97a1bc487b0d2e51a7 not found: ID does not exist"
Sep 17 11:13:39.797042 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.797019    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xb86x\" (UniqueName: \"kubernetes.io/projected/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-kube-api-access-xb86x\") pod \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") "
Sep 17 11:13:39.797148 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.797056    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-token\") pod \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") "
Sep 17 11:13:39.797148 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.797100    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-podinfo\") pod \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") "
Sep 17 11:13:39.797148 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.797144    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istiod-ca-cert\") pod \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") "
Sep 17 11:13:39.797305 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.797192    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-credential-socket\") pod \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") "
Sep 17 11:13:39.797305 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.797233    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-workload-certs\") pod \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") "
Sep 17 11:13:39.797305 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.797270    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-envoy\") pod \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") "
Sep 17 11:13:39.797453 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.797311    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-workload-socket\") pod \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") "
Sep 17 11:13:39.797453 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.797360    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-data\") pod \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\" (UID: \"bd3d0799-5524-4b05-9f2f-d0cd2a90bdac\") "
Sep 17 11:13:39.797554 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.797518    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" (UID: "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:13:39.797629 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.797593    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" (UID: "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:13:39.797807 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.797779    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-data" (OuterVolumeSpecName: "istio-data") pod "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" (UID: "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:13:39.797807 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.797792    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" (UID: "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:13:39.797935 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.797818    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" (UID: "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:13:39.798070 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.798043    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istiod-ca-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:39.798118 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.798071    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-credential-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:39.798118 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.798087    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-workload-certs\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:39.798118 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.798102    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-workload-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:39.798215 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.798115    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-data\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:39.799334 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.799310    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-kube-api-access-xb86x" (OuterVolumeSpecName: "kube-api-access-xb86x") pod "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" (UID: "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac"). InnerVolumeSpecName "kube-api-access-xb86x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:13:39.799710 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.799684    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" (UID: "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:13:39.799862 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.799832    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" (UID: "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 11:13:39.799862 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.799837    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-token" (OuterVolumeSpecName: "istio-token") pod "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" (UID: "bd3d0799-5524-4b05-9f2f-d0cd2a90bdac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:13:39.899535 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.899510    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-envoy\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:39.899535 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.899533    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xb86x\" (UniqueName: \"kubernetes.io/projected/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-kube-api-access-xb86x\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:39.899675 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.899545    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-token\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:39.899675 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:39.899553    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac-istio-podinfo\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:13:40.068054 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:40.068025    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"]
Sep 17 11:13:40.076002 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:40.075978    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-766cc3-openshift-default-8684c98854-r6bpv"]
Sep 17 11:13:41.144074 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:41.144042    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" path="/var/lib/kubelet/pods/bd3d0799-5524-4b05-9f2f-d0cd2a90bdac/volumes"
Sep 17 11:13:48.136959 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:48.136881    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:13:55.115026 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.114995    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"]
Sep 17 11:13:55.115438 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115424    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" containerName="istio-proxy"
Sep 17 11:13:55.115482 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115440    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" containerName="istio-proxy"
Sep 17 11:13:55.115482 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115449    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0365f11-a09b-4a19-be32-0323f7d0bd0e" containerName="main"
Sep 17 11:13:55.115482 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115454    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0365f11-a09b-4a19-be32-0323f7d0bd0e" containerName="main"
Sep 17 11:13:55.115482 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115466    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3730c4b-e89d-41e6-afb5-07a4465f218b" containerName="payload-pre-processing"
Sep 17 11:13:55.115482 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115471    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3730c4b-e89d-41e6-afb5-07a4465f218b" containerName="payload-pre-processing"
Sep 17 11:13:55.115657 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115489    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe" containerName="payload-processing"
Sep 17 11:13:55.115657 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115494    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe" containerName="payload-processing"
Sep 17 11:13:55.115657 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115500    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0557f635-7f09-40d7-9f65-30d31bec4730" containerName="revoke-keys"
Sep 17 11:13:55.115657 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115504    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0557f635-7f09-40d7-9f65-30d31bec4730" containerName="revoke-keys"
Sep 17 11:13:55.115657 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115510    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" containerName="istio-proxy"
Sep 17 11:13:55.115657 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115514    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" containerName="istio-proxy"
Sep 17 11:13:55.115657 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115578    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f3e730f-6ef6-42e8-8ec9-54e3c0fe00fe" containerName="payload-processing"
Sep 17 11:13:55.115657 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115589    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3730c4b-e89d-41e6-afb5-07a4465f218b" containerName="payload-pre-processing"
Sep 17 11:13:55.115657 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115594    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0365f11-a09b-4a19-be32-0323f7d0bd0e" containerName="main"
Sep 17 11:13:55.115657 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115600    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" containerName="istio-proxy"
Sep 17 11:13:55.115657 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.115606    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0557f635-7f09-40d7-9f65-30d31bec4730" containerName="revoke-keys"
Sep 17 11:13:55.117743 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.117726    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.119806 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.119784    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d078f791-openshift-default-dockercfg-jqk26\""
Sep 17 11:13:55.143795 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.143774    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"]
Sep 17 11:13:55.245665 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.245631    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-token\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.245866 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.245680    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-credential-socket\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.245866 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.245799    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istiod-ca-cert\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.245866 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.245827    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-data\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.245866 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.245849    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-workload-socket\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.246098 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.245959    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-envoy\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.246098 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.246034    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-workload-certs\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.246098 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.246062    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-podinfo\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.246098 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.246092    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwkdd\" (UniqueName: \"kubernetes.io/projected/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-kube-api-access-pwkdd\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.346811 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.346784    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istiod-ca-cert\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.346811 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.346812    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-data\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.347016 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.346831    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-workload-socket\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.347016 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.346868    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-envoy\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.347016 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.346953    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-workload-certs\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.347016 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.346977    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-podinfo\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.347016 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.347010    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwkdd\" (UniqueName: \"kubernetes.io/projected/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-kube-api-access-pwkdd\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.347249 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.347079    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-token\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.347249 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.347111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-credential-socket\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.347249 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.347238    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-data\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.347411 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.347295    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-workload-socket\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.347471 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.347421    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-workload-certs\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.347471 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.347431    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-credential-socket\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.347566 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.347531    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istiod-ca-cert\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.349290 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.349274    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-envoy\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.349604 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.349580    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-podinfo\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.358684 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.358662    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwkdd\" (UniqueName: \"kubernetes.io/projected/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-kube-api-access-pwkdd\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.358755 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.358726    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-token\") pod \"e2e-mt-d078f791-openshift-default-b9668b796-8mnd5\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") " pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.428658 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.428575    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:55.708623 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.708597    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"]
Sep 17 11:13:55.710217 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:13:55.710151    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d42fcda_751a_4f67_a43d_b63c1fe19dd7.slice/crio-8bd2339851053f3aef7d654bedca260006e818a3b28e674b10180c0b24bb9d17 WatchSource:0}: Error finding container 8bd2339851053f3aef7d654bedca260006e818a3b28e674b10180c0b24bb9d17: Status 404 returned error can't find the container with id 8bd2339851053f3aef7d654bedca260006e818a3b28e674b10180c0b24bb9d17
Sep 17 11:13:55.712577 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.712546    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:13:55.712671 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.712610    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:13:55.712671 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.712642    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:13:55.843642 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.843608    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5" event={"ID":"4d42fcda-751a-4f67-a43d-b63c1fe19dd7","Type":"ContainerStarted","Data":"e0d70385f4746ed542e1422aa9c90ba7c454597689c827d7ad1db94eeba02842"}
Sep 17 11:13:55.843818 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.843663    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5" event={"ID":"4d42fcda-751a-4f67-a43d-b63c1fe19dd7","Type":"ContainerStarted","Data":"8bd2339851053f3aef7d654bedca260006e818a3b28e674b10180c0b24bb9d17"}
Sep 17 11:13:55.867427 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:55.867371    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5" podStartSLOduration=0.867354827 podStartE2EDuration="867.354827ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:13:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:13:55.863663262 +0000 UTC m=+1231.180026419" watchObservedRunningTime="2026-09-17 11:13:55.867354827 +0000 UTC m=+1231.183717981"
Sep 17 11:13:56.429658 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:56.429631    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:56.435049 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:56.435026    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:56.850995 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:56.850966    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:13:56.851980 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:13:56.851967    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:14:02.137492 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:02.137453    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:14:21.172747 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:21.172705    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67"]
Sep 17 11:14:21.173404 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:21.173383    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd3d0799-5524-4b05-9f2f-d0cd2a90bdac" containerName="istio-proxy"
Sep 17 11:14:21.175495 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:21.175474    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67"
Sep 17 11:14:21.178102 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:21.178071    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:14:21.178102 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:21.178087    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:14:21.186042 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:21.186019    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67"]
Sep 17 11:14:21.292499 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:21.292463    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1f1e0b9-89b0-490b-b42f-20b1822021a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67\" (UID: \"e1f1e0b9-89b0-490b-b42f-20b1822021a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67"
Sep 17 11:14:21.393675 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:21.393624    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1f1e0b9-89b0-490b-b42f-20b1822021a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67\" (UID: \"e1f1e0b9-89b0-490b-b42f-20b1822021a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67"
Sep 17 11:14:21.394480 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:21.394455    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1f1e0b9-89b0-490b-b42f-20b1822021a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67\" (UID: \"e1f1e0b9-89b0-490b-b42f-20b1822021a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67"
Sep 17 11:14:21.490396 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:21.490365    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67"
Sep 17 11:14:21.785999 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:21.785968    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67"]
Sep 17 11:14:21.787704 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:14:21.787666    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f1e0b9_89b0_490b_b42f_20b1822021a3.slice/crio-3634c0564d3b599faa5e6b193e5c02b103c23e8ec57c86fa57bb8a90344d43bd WatchSource:0}: Error finding container 3634c0564d3b599faa5e6b193e5c02b103c23e8ec57c86fa57bb8a90344d43bd: Status 404 returned error can't find the container with id 3634c0564d3b599faa5e6b193e5c02b103c23e8ec57c86fa57bb8a90344d43bd
Sep 17 11:14:21.997775 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:21.997742    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67" event={"ID":"e1f1e0b9-89b0-490b-b42f-20b1822021a3","Type":"ContainerStarted","Data":"3634c0564d3b599faa5e6b193e5c02b103c23e8ec57c86fa57bb8a90344d43bd"}
Sep 17 11:14:23.006283 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:23.006256    2564 generic.go:356] "Generic (PLEG): container finished" podID="e1f1e0b9-89b0-490b-b42f-20b1822021a3" containerID="6faadcf155286e9eaba9fb8832bb5a1f51f98ec529dbc51b747c46f6d7cb2bc9" exitCode=0
Sep 17 11:14:23.006660 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:23.006340    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67" event={"ID":"e1f1e0b9-89b0-490b-b42f-20b1822021a3","Type":"ContainerDied","Data":"6faadcf155286e9eaba9fb8832bb5a1f51f98ec529dbc51b747c46f6d7cb2bc9"}
Sep 17 11:14:24.014436 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:24.014406    2564 generic.go:356] "Generic (PLEG): container finished" podID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" containerID="e0d70385f4746ed542e1422aa9c90ba7c454597689c827d7ad1db94eeba02842" exitCode=0
Sep 17 11:14:24.014801 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:24.014484    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5" event={"ID":"4d42fcda-751a-4f67-a43d-b63c1fe19dd7","Type":"ContainerDied","Data":"e0d70385f4746ed542e1422aa9c90ba7c454597689c827d7ad1db94eeba02842"}
Sep 17 11:14:24.015134 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:24.015095    2564 scope.go:117] "RemoveContainer" containerID="e0d70385f4746ed542e1422aa9c90ba7c454597689c827d7ad1db94eeba02842"
Sep 17 11:14:24.016364 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:24.016333    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:14:24.016446 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:24.016409    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:14:24.016446 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:24.016442    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:14:24.168481 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:24.168459    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67"
Sep 17 11:14:24.321050 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:24.320969    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1f1e0b9-89b0-490b-b42f-20b1822021a3-maas-api-service-ca\") pod \"e1f1e0b9-89b0-490b-b42f-20b1822021a3\" (UID: \"e1f1e0b9-89b0-490b-b42f-20b1822021a3\") "
Sep 17 11:14:24.321291 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:24.321270    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1f1e0b9-89b0-490b-b42f-20b1822021a3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e1f1e0b9-89b0-490b-b42f-20b1822021a3" (UID: "e1f1e0b9-89b0-490b-b42f-20b1822021a3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:14:24.422133 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:24.422100    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1f1e0b9-89b0-490b-b42f-20b1822021a3-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:25.022767 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:25.022736    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67"
Sep 17 11:14:25.022767 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:25.022756    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67" event={"ID":"e1f1e0b9-89b0-490b-b42f-20b1822021a3","Type":"ContainerDied","Data":"3634c0564d3b599faa5e6b193e5c02b103c23e8ec57c86fa57bb8a90344d43bd"}
Sep 17 11:14:25.023335 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:25.022807    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3634c0564d3b599faa5e6b193e5c02b103c23e8ec57c86fa57bb8a90344d43bd"
Sep 17 11:14:25.025185 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:25.025157    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5" event={"ID":"4d42fcda-751a-4f67-a43d-b63c1fe19dd7","Type":"ContainerStarted","Data":"911329ad1ef5b15edc2ac643c4938fb1c64013b3d29e16ae925243cb179aec03"}
Sep 17 11:14:25.429652 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:25.429613    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:14:25.431153 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:25.431124    2564 patch_prober.go:28] interesting pod/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 17 11:14:25.431267 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:25.431178    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5" podUID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 17 11:14:26.032556 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:26.032523    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"]
Sep 17 11:14:26.066705 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:26.066673    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"]
Sep 17 11:14:26.429452 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:26.429351    2564 patch_prober.go:28] interesting pod/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 17 11:14:26.429452 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:26.429418    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5" podUID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 17 11:14:26.851885 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:26.851848    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:14:27.429418 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:27.429378    2564 patch_prober.go:28] interesting pod/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 17 11:14:27.429812 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:27.429438    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5" podUID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 17 11:14:28.429492 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:28.429456    2564 patch_prober.go:28] interesting pod/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 17 11:14:28.429866 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:28.429508    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5" podUID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 17 11:14:29.429250 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:29.429210    2564 patch_prober.go:28] interesting pod/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 17 11:14:29.429424 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:29.429270    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5" podUID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 17 11:14:30.429755 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:30.429714    2564 patch_prober.go:28] interesting pod/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 17 11:14:30.430170 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:30.429770    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5" podUID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 17 11:14:31.429235 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:31.429199    2564 patch_prober.go:28] interesting pod/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 17 11:14:31.429436 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:31.429275    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5" podUID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 17 11:14:32.434020 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:32.433987    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:14:32.434892 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:32.434876    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:14:35.510138 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:35.510104    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d078f791-8nxpl"]
Sep 17 11:14:35.510527 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:35.510512    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1f1e0b9-89b0-490b-b42f-20b1822021a3" containerName="revoke-keys"
Sep 17 11:14:35.510575 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:35.510531    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1f1e0b9-89b0-490b-b42f-20b1822021a3" containerName="revoke-keys"
Sep 17 11:14:35.510622 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:35.510613    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1f1e0b9-89b0-490b-b42f-20b1822021a3" containerName="revoke-keys"
Sep 17 11:14:35.513549 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:35.513533    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d078f791-8nxpl"
Sep 17 11:14:35.515651 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:35.515633    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:14:35.515812 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:35.515795    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:14:35.521128 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:35.520793    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d078f791-8nxpl"]
Sep 17 11:14:35.633840 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:35.633800    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/196dc8c9-1a1c-4780-a26e-45f48dcea8e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d078f791-8nxpl\" (UID: \"196dc8c9-1a1c-4780-a26e-45f48dcea8e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d078f791-8nxpl"
Sep 17 11:14:35.735264 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:35.735208    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/196dc8c9-1a1c-4780-a26e-45f48dcea8e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d078f791-8nxpl\" (UID: \"196dc8c9-1a1c-4780-a26e-45f48dcea8e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d078f791-8nxpl"
Sep 17 11:14:35.735841 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:35.735820    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/196dc8c9-1a1c-4780-a26e-45f48dcea8e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d078f791-8nxpl\" (UID: \"196dc8c9-1a1c-4780-a26e-45f48dcea8e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d078f791-8nxpl"
Sep 17 11:14:35.825044 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:35.824957    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d078f791-8nxpl"
Sep 17 11:14:36.179067 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:36.179039    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d078f791-8nxpl"]
Sep 17 11:14:36.180026 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:14:36.180003    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod196dc8c9_1a1c_4780_a26e_45f48dcea8e7.slice/crio-443539df5d0eea967a47583b44edf95db26ed80b9ae66b220ddd6b589d63e856 WatchSource:0}: Error finding container 443539df5d0eea967a47583b44edf95db26ed80b9ae66b220ddd6b589d63e856: Status 404 returned error can't find the container with id 443539df5d0eea967a47583b44edf95db26ed80b9ae66b220ddd6b589d63e856
Sep 17 11:14:36.916094 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:36.916063    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"]
Sep 17 11:14:36.916428 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:36.916303    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx" podUID="f30081de-2cd4-4b52-8379-8c4c5021477e" containerName="payload-processing" containerID="cri-o://8d7896d83273710fc73500c902035e1449df5abf49099129b1ab8062894172b7" gracePeriod=30
Sep 17 11:14:37.099763 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:37.099729    2564 generic.go:356] "Generic (PLEG): container finished" podID="f30081de-2cd4-4b52-8379-8c4c5021477e" containerID="8d7896d83273710fc73500c902035e1449df5abf49099129b1ab8062894172b7" exitCode=0
Sep 17 11:14:37.100143 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:37.099770    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx" event={"ID":"f30081de-2cd4-4b52-8379-8c4c5021477e","Type":"ContainerDied","Data":"8d7896d83273710fc73500c902035e1449df5abf49099129b1ab8062894172b7"}
Sep 17 11:14:37.102934 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:37.102470    2564 generic.go:356] "Generic (PLEG): container finished" podID="196dc8c9-1a1c-4780-a26e-45f48dcea8e7" containerID="90c0715e0fe0572b1ed516f8c7ee9ab08e6fad70c05aacb5505a5896c58bac65" exitCode=0
Sep 17 11:14:37.102934 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:37.102547    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d078f791-8nxpl" event={"ID":"196dc8c9-1a1c-4780-a26e-45f48dcea8e7","Type":"ContainerDied","Data":"90c0715e0fe0572b1ed516f8c7ee9ab08e6fad70c05aacb5505a5896c58bac65"}
Sep 17 11:14:37.102934 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:37.102590    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d078f791-8nxpl" event={"ID":"196dc8c9-1a1c-4780-a26e-45f48dcea8e7","Type":"ContainerStarted","Data":"443539df5d0eea967a47583b44edf95db26ed80b9ae66b220ddd6b589d63e856"}
Sep 17 11:14:37.109842 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:37.109818    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67"]
Sep 17 11:14:37.116982 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:37.116960    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-d7b97f-rwb67"]
Sep 17 11:14:37.144554 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:37.144528    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1f1e0b9-89b0-490b-b42f-20b1822021a3" path="/var/lib/kubelet/pods/e1f1e0b9-89b0-490b-b42f-20b1822021a3/volumes"
Sep 17 11:14:37.175681 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:37.175655    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"
Sep 17 11:14:37.351911 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:37.351848    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f30081de-2cd4-4b52-8379-8c4c5021477e-plugins-config-volume\") pod \"f30081de-2cd4-4b52-8379-8c4c5021477e\" (UID: \"f30081de-2cd4-4b52-8379-8c4c5021477e\") "
Sep 17 11:14:37.352142 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:37.351969    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gx9w\" (UniqueName: \"kubernetes.io/projected/f30081de-2cd4-4b52-8379-8c4c5021477e-kube-api-access-2gx9w\") pod \"f30081de-2cd4-4b52-8379-8c4c5021477e\" (UID: \"f30081de-2cd4-4b52-8379-8c4c5021477e\") "
Sep 17 11:14:37.352308 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:37.352288    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f30081de-2cd4-4b52-8379-8c4c5021477e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f30081de-2cd4-4b52-8379-8c4c5021477e" (UID: "f30081de-2cd4-4b52-8379-8c4c5021477e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:14:37.354202 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:37.354170    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f30081de-2cd4-4b52-8379-8c4c5021477e-kube-api-access-2gx9w" (OuterVolumeSpecName: "kube-api-access-2gx9w") pod "f30081de-2cd4-4b52-8379-8c4c5021477e" (UID: "f30081de-2cd4-4b52-8379-8c4c5021477e"). InnerVolumeSpecName "kube-api-access-2gx9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:14:37.453642 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:37.453562    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f30081de-2cd4-4b52-8379-8c4c5021477e-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:37.453642 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:37.453591    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gx9w\" (UniqueName: \"kubernetes.io/projected/f30081de-2cd4-4b52-8379-8c4c5021477e-kube-api-access-2gx9w\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:38.113889 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:38.113855    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"
Sep 17 11:14:38.114272 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:38.113882    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx" event={"ID":"f30081de-2cd4-4b52-8379-8c4c5021477e","Type":"ContainerDied","Data":"89601f2aac98151a2ccb44d58ae9db4f9cd0f9b507120e1cdd11257cc1c91cfb"}
Sep 17 11:14:38.114272 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:38.114009    2564 scope.go:117] "RemoveContainer" containerID="8d7896d83273710fc73500c902035e1449df5abf49099129b1ab8062894172b7"
Sep 17 11:14:38.143644 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:38.143611    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"]
Sep 17 11:14:38.146867 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:38.146830    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-d7b97f-6c47b7d997-lwvsx"]
Sep 17 11:14:38.250810 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:38.250790    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d078f791-8nxpl"
Sep 17 11:14:38.362443 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:38.362415    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/196dc8c9-1a1c-4780-a26e-45f48dcea8e7-maas-api-service-ca\") pod \"196dc8c9-1a1c-4780-a26e-45f48dcea8e7\" (UID: \"196dc8c9-1a1c-4780-a26e-45f48dcea8e7\") "
Sep 17 11:14:38.362815 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:38.362794    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/196dc8c9-1a1c-4780-a26e-45f48dcea8e7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "196dc8c9-1a1c-4780-a26e-45f48dcea8e7" (UID: "196dc8c9-1a1c-4780-a26e-45f48dcea8e7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:14:38.463644 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:38.463562    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/196dc8c9-1a1c-4780-a26e-45f48dcea8e7-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:39.123164 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:39.123131    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d078f791-8nxpl"
Sep 17 11:14:39.123546 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:39.123174    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d078f791-8nxpl" event={"ID":"196dc8c9-1a1c-4780-a26e-45f48dcea8e7","Type":"ContainerDied","Data":"443539df5d0eea967a47583b44edf95db26ed80b9ae66b220ddd6b589d63e856"}
Sep 17 11:14:39.123546 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:39.123234    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="443539df5d0eea967a47583b44edf95db26ed80b9ae66b220ddd6b589d63e856"
Sep 17 11:14:39.146915 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:39.146873    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f30081de-2cd4-4b52-8379-8c4c5021477e" path="/var/lib/kubelet/pods/f30081de-2cd4-4b52-8379-8c4c5021477e/volumes"
Sep 17 11:14:41.052238 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.052176    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp" podUID="604b7a3a-b16d-43c1-ad03-71fff923c079" containerName="main" containerID="cri-o://2cf6e8bbd477b80845a6aea4670029c6589f96c1e30f6b9e8446a9bbc1fbe20f" gracePeriod=45
Sep 17 11:14:41.086120 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.086054    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k" podUID="cd7090ce-fe70-462c-85bc-c5e4a15840e1" containerName="main" containerID="cri-o://934996d524c6c785d5fd4a024bfe1135770c5ba2009e8579beee260dedf5528a" gracePeriod=45
Sep 17 11:14:41.359045 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.359018    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:14:41.362133 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.362114    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:14:41.391715 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.391618    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd7090ce-fe70-462c-85bc-c5e4a15840e1-tls-certs\") pod \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") "
Sep 17 11:14:41.391715 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.391699    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-model-cache\") pod \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") "
Sep 17 11:14:41.392674 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.391731    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-dshm\") pod \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") "
Sep 17 11:14:41.392674 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.391771    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-home\") pod \"604b7a3a-b16d-43c1-ad03-71fff923c079\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") "
Sep 17 11:14:41.392674 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.391797    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/604b7a3a-b16d-43c1-ad03-71fff923c079-tls-certs\") pod \"604b7a3a-b16d-43c1-ad03-71fff923c079\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") "
Sep 17 11:14:41.392674 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.391829    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-tmp-dir\") pod \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") "
Sep 17 11:14:41.392674 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.391869    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-model-cache\") pod \"604b7a3a-b16d-43c1-ad03-71fff923c079\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") "
Sep 17 11:14:41.392674 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.391932    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-tmp-dir\") pod \"604b7a3a-b16d-43c1-ad03-71fff923c079\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") "
Sep 17 11:14:41.392674 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.391957    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-home\") pod \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") "
Sep 17 11:14:41.392674 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.391988    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ncss\" (UniqueName: \"kubernetes.io/projected/cd7090ce-fe70-462c-85bc-c5e4a15840e1-kube-api-access-6ncss\") pod \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\" (UID: \"cd7090ce-fe70-462c-85bc-c5e4a15840e1\") "
Sep 17 11:14:41.392674 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.392049    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xppdg\" (UniqueName: \"kubernetes.io/projected/604b7a3a-b16d-43c1-ad03-71fff923c079-kube-api-access-xppdg\") pod \"604b7a3a-b16d-43c1-ad03-71fff923c079\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") "
Sep 17 11:14:41.392674 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.392069    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-dshm\") pod \"604b7a3a-b16d-43c1-ad03-71fff923c079\" (UID: \"604b7a3a-b16d-43c1-ad03-71fff923c079\") "
Sep 17 11:14:41.393221 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.392856    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-model-cache" (OuterVolumeSpecName: "model-cache") pod "cd7090ce-fe70-462c-85bc-c5e4a15840e1" (UID: "cd7090ce-fe70-462c-85bc-c5e4a15840e1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:14:41.394883 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.394033    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-home" (OuterVolumeSpecName: "home") pod "604b7a3a-b16d-43c1-ad03-71fff923c079" (UID: "604b7a3a-b16d-43c1-ad03-71fff923c079"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:14:41.394883 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.394306    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cd7090ce-fe70-462c-85bc-c5e4a15840e1" (UID: "cd7090ce-fe70-462c-85bc-c5e4a15840e1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:14:41.394883 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.394482    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-home" (OuterVolumeSpecName: "home") pod "cd7090ce-fe70-462c-85bc-c5e4a15840e1" (UID: "cd7090ce-fe70-462c-85bc-c5e4a15840e1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:14:41.394883 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.394645    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "604b7a3a-b16d-43c1-ad03-71fff923c079" (UID: "604b7a3a-b16d-43c1-ad03-71fff923c079"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:14:41.395464 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.395436    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-model-cache" (OuterVolumeSpecName: "model-cache") pod "604b7a3a-b16d-43c1-ad03-71fff923c079" (UID: "604b7a3a-b16d-43c1-ad03-71fff923c079"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:14:41.395997 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.395972    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-dshm" (OuterVolumeSpecName: "dshm") pod "604b7a3a-b16d-43c1-ad03-71fff923c079" (UID: "604b7a3a-b16d-43c1-ad03-71fff923c079"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:14:41.396178 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.396132    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd7090ce-fe70-462c-85bc-c5e4a15840e1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cd7090ce-fe70-462c-85bc-c5e4a15840e1" (UID: "cd7090ce-fe70-462c-85bc-c5e4a15840e1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 11:14:41.398311 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.398187    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/604b7a3a-b16d-43c1-ad03-71fff923c079-kube-api-access-xppdg" (OuterVolumeSpecName: "kube-api-access-xppdg") pod "604b7a3a-b16d-43c1-ad03-71fff923c079" (UID: "604b7a3a-b16d-43c1-ad03-71fff923c079"). InnerVolumeSpecName "kube-api-access-xppdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:14:41.399044 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.398665    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/604b7a3a-b16d-43c1-ad03-71fff923c079-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "604b7a3a-b16d-43c1-ad03-71fff923c079" (UID: "604b7a3a-b16d-43c1-ad03-71fff923c079"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 11:14:41.399044 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.398671    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd7090ce-fe70-462c-85bc-c5e4a15840e1-kube-api-access-6ncss" (OuterVolumeSpecName: "kube-api-access-6ncss") pod "cd7090ce-fe70-462c-85bc-c5e4a15840e1" (UID: "cd7090ce-fe70-462c-85bc-c5e4a15840e1"). InnerVolumeSpecName "kube-api-access-6ncss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:14:41.399984 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.399950    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-dshm" (OuterVolumeSpecName: "dshm") pod "cd7090ce-fe70-462c-85bc-c5e4a15840e1" (UID: "cd7090ce-fe70-462c-85bc-c5e4a15840e1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:14:41.493296 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.493259    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd7090ce-fe70-462c-85bc-c5e4a15840e1-tls-certs\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:41.493296 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.493290    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-model-cache\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:41.493296 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.493301    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-dshm\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:41.493539 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.493312    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-home\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:41.493539 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.493322    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/604b7a3a-b16d-43c1-ad03-71fff923c079-tls-certs\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:41.493539 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.493331    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-tmp-dir\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:41.493539 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.493342    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-model-cache\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:41.493539 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.493351    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-tmp-dir\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:41.493539 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.493361    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd7090ce-fe70-462c-85bc-c5e4a15840e1-home\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:41.493539 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.493373    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ncss\" (UniqueName: \"kubernetes.io/projected/cd7090ce-fe70-462c-85bc-c5e4a15840e1-kube-api-access-6ncss\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:41.493539 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.493385    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xppdg\" (UniqueName: \"kubernetes.io/projected/604b7a3a-b16d-43c1-ad03-71fff923c079-kube-api-access-xppdg\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:41.493539 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:41.493397    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/604b7a3a-b16d-43c1-ad03-71fff923c079-dshm\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:42.144315 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.144278    2564 generic.go:356] "Generic (PLEG): container finished" podID="cd7090ce-fe70-462c-85bc-c5e4a15840e1" containerID="934996d524c6c785d5fd4a024bfe1135770c5ba2009e8579beee260dedf5528a" exitCode=0
Sep 17 11:14:42.144770 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.144325    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"
Sep 17 11:14:42.144770 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.144356    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k" event={"ID":"cd7090ce-fe70-462c-85bc-c5e4a15840e1","Type":"ContainerDied","Data":"934996d524c6c785d5fd4a024bfe1135770c5ba2009e8579beee260dedf5528a"}
Sep 17 11:14:42.144770 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.144398    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k" event={"ID":"cd7090ce-fe70-462c-85bc-c5e4a15840e1","Type":"ContainerDied","Data":"7b3a3d16eabf47007a1c7b7ea6ea4ef4fdc6eaca59300c07f3d05b2d28189355"}
Sep 17 11:14:42.144770 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.144419    2564 scope.go:117] "RemoveContainer" containerID="934996d524c6c785d5fd4a024bfe1135770c5ba2009e8579beee260dedf5528a"
Sep 17 11:14:42.145939 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.145914    2564 generic.go:356] "Generic (PLEG): container finished" podID="604b7a3a-b16d-43c1-ad03-71fff923c079" containerID="2cf6e8bbd477b80845a6aea4670029c6589f96c1e30f6b9e8446a9bbc1fbe20f" exitCode=0
Sep 17 11:14:42.146000 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.145951    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp" event={"ID":"604b7a3a-b16d-43c1-ad03-71fff923c079","Type":"ContainerDied","Data":"2cf6e8bbd477b80845a6aea4670029c6589f96c1e30f6b9e8446a9bbc1fbe20f"}
Sep 17 11:14:42.146051 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.146012    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp" event={"ID":"604b7a3a-b16d-43c1-ad03-71fff923c079","Type":"ContainerDied","Data":"4ccdc0f1da9ea41ad0bde0bb4e7205e717bc5d40cebc3791acbea0acbf4a3a77"}
Sep 17 11:14:42.146101 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.145964    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"
Sep 17 11:14:42.155042 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.155025    2564 scope.go:117] "RemoveContainer" containerID="934996d524c6c785d5fd4a024bfe1135770c5ba2009e8579beee260dedf5528a"
Sep 17 11:14:42.155334 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:14:42.155310    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"934996d524c6c785d5fd4a024bfe1135770c5ba2009e8579beee260dedf5528a\": container with ID starting with 934996d524c6c785d5fd4a024bfe1135770c5ba2009e8579beee260dedf5528a not found: ID does not exist" containerID="934996d524c6c785d5fd4a024bfe1135770c5ba2009e8579beee260dedf5528a"
Sep 17 11:14:42.155425 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.155347    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"934996d524c6c785d5fd4a024bfe1135770c5ba2009e8579beee260dedf5528a"} err="failed to get container status \"934996d524c6c785d5fd4a024bfe1135770c5ba2009e8579beee260dedf5528a\": rpc error: code = NotFound desc = could not find container \"934996d524c6c785d5fd4a024bfe1135770c5ba2009e8579beee260dedf5528a\": container with ID starting with 934996d524c6c785d5fd4a024bfe1135770c5ba2009e8579beee260dedf5528a not found: ID does not exist"
Sep 17 11:14:42.155425 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.155371    2564 scope.go:117] "RemoveContainer" containerID="2cf6e8bbd477b80845a6aea4670029c6589f96c1e30f6b9e8446a9bbc1fbe20f"
Sep 17 11:14:42.163963 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.163946    2564 scope.go:117] "RemoveContainer" containerID="2cf6e8bbd477b80845a6aea4670029c6589f96c1e30f6b9e8446a9bbc1fbe20f"
Sep 17 11:14:42.164195 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:14:42.164176    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cf6e8bbd477b80845a6aea4670029c6589f96c1e30f6b9e8446a9bbc1fbe20f\": container with ID starting with 2cf6e8bbd477b80845a6aea4670029c6589f96c1e30f6b9e8446a9bbc1fbe20f not found: ID does not exist" containerID="2cf6e8bbd477b80845a6aea4670029c6589f96c1e30f6b9e8446a9bbc1fbe20f"
Sep 17 11:14:42.164246 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.164203    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cf6e8bbd477b80845a6aea4670029c6589f96c1e30f6b9e8446a9bbc1fbe20f"} err="failed to get container status \"2cf6e8bbd477b80845a6aea4670029c6589f96c1e30f6b9e8446a9bbc1fbe20f\": rpc error: code = NotFound desc = could not find container \"2cf6e8bbd477b80845a6aea4670029c6589f96c1e30f6b9e8446a9bbc1fbe20f\": container with ID starting with 2cf6e8bbd477b80845a6aea4670029c6589f96c1e30f6b9e8446a9bbc1fbe20f not found: ID does not exist"
Sep 17 11:14:42.170884 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.170864    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"]
Sep 17 11:14:42.175977 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.175956    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-d7b97f/simulator-w0-d7b97f-kserve-7df974d554-gh98k"]
Sep 17 11:14:42.186778 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.186753    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"]
Sep 17 11:14:42.193237 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:42.193219    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-d7b97f/distinct-w0-d7b97f-kserve-775459d6f4-6fccp"]
Sep 17 11:14:43.144144 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:43.144117    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="604b7a3a-b16d-43c1-ad03-71fff923c079" path="/var/lib/kubelet/pods/604b7a3a-b16d-43c1-ad03-71fff923c079/volumes"
Sep 17 11:14:43.144520 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:43.144508    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd7090ce-fe70-462c-85bc-c5e4a15840e1" path="/var/lib/kubelet/pods/cd7090ce-fe70-462c-85bc-c5e4a15840e1/volumes"
Sep 17 11:14:46.829053 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:46.829011    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d078f791-8nxpl"]
Sep 17 11:14:46.837808 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:46.837774    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d078f791-8nxpl"]
Sep 17 11:14:47.143053 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:47.142973    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="196dc8c9-1a1c-4780-a26e-45f48dcea8e7" path="/var/lib/kubelet/pods/196dc8c9-1a1c-4780-a26e-45f48dcea8e7/volumes"
Sep 17 11:14:51.137149 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:51.137115    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:14:53.692090 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:53.692048    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"]
Sep 17 11:14:53.692578 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:53.692281    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5" podUID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" containerName="istio-proxy" containerID="cri-o://911329ad1ef5b15edc2ac643c4938fb1c64013b3d29e16ae925243cb179aec03" gracePeriod=30
Sep 17 11:14:58.952342 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:58.952318    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:14:59.053706 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.053669    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-data\") pod \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") "
Sep 17 11:14:59.053706 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.053715    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-token\") pod \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") "
Sep 17 11:14:59.054006 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.053769    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-podinfo\") pod \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") "
Sep 17 11:14:59.054006 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.053811    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwkdd\" (UniqueName: \"kubernetes.io/projected/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-kube-api-access-pwkdd\") pod \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") "
Sep 17 11:14:59.054006 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.053837    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-envoy\") pod \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") "
Sep 17 11:14:59.054006 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.053865    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istiod-ca-cert\") pod \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") "
Sep 17 11:14:59.054006 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.053936    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-workload-certs\") pod \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") "
Sep 17 11:14:59.054006 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.053984    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-credential-socket\") pod \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") "
Sep 17 11:14:59.054320 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.054005    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-data" (OuterVolumeSpecName: "istio-data") pod "4d42fcda-751a-4f67-a43d-b63c1fe19dd7" (UID: "4d42fcda-751a-4f67-a43d-b63c1fe19dd7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:14:59.054320 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.054079    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-workload-socket\") pod \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\" (UID: \"4d42fcda-751a-4f67-a43d-b63c1fe19dd7\") "
Sep 17 11:14:59.054320 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.054244    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4d42fcda-751a-4f67-a43d-b63c1fe19dd7" (UID: "4d42fcda-751a-4f67-a43d-b63c1fe19dd7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:14:59.054486 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.054400    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4d42fcda-751a-4f67-a43d-b63c1fe19dd7" (UID: "4d42fcda-751a-4f67-a43d-b63c1fe19dd7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:14:59.054486 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.054428    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4d42fcda-751a-4f67-a43d-b63c1fe19dd7" (UID: "4d42fcda-751a-4f67-a43d-b63c1fe19dd7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:14:59.054486 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.054442    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4d42fcda-751a-4f67-a43d-b63c1fe19dd7" (UID: "4d42fcda-751a-4f67-a43d-b63c1fe19dd7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:14:59.054704 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.054495    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-credential-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:59.054704 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.054515    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-data\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:59.054704 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.054529    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-workload-certs\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:59.056245 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.056217    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4d42fcda-751a-4f67-a43d-b63c1fe19dd7" (UID: "4d42fcda-751a-4f67-a43d-b63c1fe19dd7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 11:14:59.056366 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.056241    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-kube-api-access-pwkdd" (OuterVolumeSpecName: "kube-api-access-pwkdd") pod "4d42fcda-751a-4f67-a43d-b63c1fe19dd7" (UID: "4d42fcda-751a-4f67-a43d-b63c1fe19dd7"). InnerVolumeSpecName "kube-api-access-pwkdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:14:59.056528 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.056502    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4d42fcda-751a-4f67-a43d-b63c1fe19dd7" (UID: "4d42fcda-751a-4f67-a43d-b63c1fe19dd7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:14:59.056528 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.056518    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-token" (OuterVolumeSpecName: "istio-token") pod "4d42fcda-751a-4f67-a43d-b63c1fe19dd7" (UID: "4d42fcda-751a-4f67-a43d-b63c1fe19dd7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:14:59.154878 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.154854    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-podinfo\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:59.154878 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.154878    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwkdd\" (UniqueName: \"kubernetes.io/projected/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-kube-api-access-pwkdd\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:59.155055 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.154893    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-envoy\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:59.155055 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.154922    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istiod-ca-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:59.155055 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.154936    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-workload-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:59.155055 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.154948    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d42fcda-751a-4f67-a43d-b63c1fe19dd7-istio-token\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:14:59.172014 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:14:59.171990    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d42fcda_751a_4f67_a43d_b63c1fe19dd7.slice\": RecentStats: unable to find data in memory cache]"
Sep 17 11:14:59.246522 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.246497    2564 generic.go:356] "Generic (PLEG): container finished" podID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" containerID="911329ad1ef5b15edc2ac643c4938fb1c64013b3d29e16ae925243cb179aec03" exitCode=0
Sep 17 11:14:59.246632 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.246544    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"
Sep 17 11:14:59.246632 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.246574    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5" event={"ID":"4d42fcda-751a-4f67-a43d-b63c1fe19dd7","Type":"ContainerDied","Data":"911329ad1ef5b15edc2ac643c4938fb1c64013b3d29e16ae925243cb179aec03"}
Sep 17 11:14:59.246632 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.246621    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5" event={"ID":"4d42fcda-751a-4f67-a43d-b63c1fe19dd7","Type":"ContainerDied","Data":"8bd2339851053f3aef7d654bedca260006e818a3b28e674b10180c0b24bb9d17"}
Sep 17 11:14:59.246738 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.246641    2564 scope.go:117] "RemoveContainer" containerID="911329ad1ef5b15edc2ac643c4938fb1c64013b3d29e16ae925243cb179aec03"
Sep 17 11:14:59.256046 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.256030    2564 scope.go:117] "RemoveContainer" containerID="e0d70385f4746ed542e1422aa9c90ba7c454597689c827d7ad1db94eeba02842"
Sep 17 11:14:59.264375 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.264355    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"]
Sep 17 11:14:59.265019 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.265000    2564 scope.go:117] "RemoveContainer" containerID="911329ad1ef5b15edc2ac643c4938fb1c64013b3d29e16ae925243cb179aec03"
Sep 17 11:14:59.265298 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:14:59.265282    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"911329ad1ef5b15edc2ac643c4938fb1c64013b3d29e16ae925243cb179aec03\": container with ID starting with 911329ad1ef5b15edc2ac643c4938fb1c64013b3d29e16ae925243cb179aec03 not found: ID does not exist" containerID="911329ad1ef5b15edc2ac643c4938fb1c64013b3d29e16ae925243cb179aec03"
Sep 17 11:14:59.265362 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.265305    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"911329ad1ef5b15edc2ac643c4938fb1c64013b3d29e16ae925243cb179aec03"} err="failed to get container status \"911329ad1ef5b15edc2ac643c4938fb1c64013b3d29e16ae925243cb179aec03\": rpc error: code = NotFound desc = could not find container \"911329ad1ef5b15edc2ac643c4938fb1c64013b3d29e16ae925243cb179aec03\": container with ID starting with 911329ad1ef5b15edc2ac643c4938fb1c64013b3d29e16ae925243cb179aec03 not found: ID does not exist"
Sep 17 11:14:59.265362 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.265321    2564 scope.go:117] "RemoveContainer" containerID="e0d70385f4746ed542e1422aa9c90ba7c454597689c827d7ad1db94eeba02842"
Sep 17 11:14:59.265558 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:14:59.265540    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0d70385f4746ed542e1422aa9c90ba7c454597689c827d7ad1db94eeba02842\": container with ID starting with e0d70385f4746ed542e1422aa9c90ba7c454597689c827d7ad1db94eeba02842 not found: ID does not exist" containerID="e0d70385f4746ed542e1422aa9c90ba7c454597689c827d7ad1db94eeba02842"
Sep 17 11:14:59.265592 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.265563    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0d70385f4746ed542e1422aa9c90ba7c454597689c827d7ad1db94eeba02842"} err="failed to get container status \"e0d70385f4746ed542e1422aa9c90ba7c454597689c827d7ad1db94eeba02842\": rpc error: code = NotFound desc = could not find container \"e0d70385f4746ed542e1422aa9c90ba7c454597689c827d7ad1db94eeba02842\": container with ID starting with e0d70385f4746ed542e1422aa9c90ba7c454597689c827d7ad1db94eeba02842 not found: ID does not exist"
Sep 17 11:14:59.269372 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:14:59.269353    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d078f791-openshift-default-b9668b796-8mnd5"]
Sep 17 11:15:00.147500 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.147463    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29827395-wxbtr"]
Sep 17 11:15:00.148025 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.147927    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd7090ce-fe70-462c-85bc-c5e4a15840e1" containerName="main"
Sep 17 11:15:00.148025 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.147941    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd7090ce-fe70-462c-85bc-c5e4a15840e1" containerName="main"
Sep 17 11:15:00.148025 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.147958    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" containerName="istio-proxy"
Sep 17 11:15:00.148025 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.147963    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" containerName="istio-proxy"
Sep 17 11:15:00.148025 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.147974    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="604b7a3a-b16d-43c1-ad03-71fff923c079" containerName="main"
Sep 17 11:15:00.148025 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.147981    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="604b7a3a-b16d-43c1-ad03-71fff923c079" containerName="main"
Sep 17 11:15:00.148025 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.147988    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" containerName="istio-proxy"
Sep 17 11:15:00.148025 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.147993    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" containerName="istio-proxy"
Sep 17 11:15:00.148025 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.148003    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f30081de-2cd4-4b52-8379-8c4c5021477e" containerName="payload-processing"
Sep 17 11:15:00.148025 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.148008    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f30081de-2cd4-4b52-8379-8c4c5021477e" containerName="payload-processing"
Sep 17 11:15:00.148025 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.148014    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="196dc8c9-1a1c-4780-a26e-45f48dcea8e7" containerName="revoke-keys"
Sep 17 11:15:00.148025 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.148019    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="196dc8c9-1a1c-4780-a26e-45f48dcea8e7" containerName="revoke-keys"
Sep 17 11:15:00.148657 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.148115    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd7090ce-fe70-462c-85bc-c5e4a15840e1" containerName="main"
Sep 17 11:15:00.148657 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.148143    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" containerName="istio-proxy"
Sep 17 11:15:00.148657 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.148155    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="196dc8c9-1a1c-4780-a26e-45f48dcea8e7" containerName="revoke-keys"
Sep 17 11:15:00.148657 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.148164    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f30081de-2cd4-4b52-8379-8c4c5021477e" containerName="payload-processing"
Sep 17 11:15:00.148657 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.148173    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="604b7a3a-b16d-43c1-ad03-71fff923c079" containerName="main"
Sep 17 11:15:00.152778 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.152760    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-026f86-29827395-z5hdk"]
Sep 17 11:15:00.152944 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.152924    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827395-wxbtr"
Sep 17 11:15:00.153184 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.153167    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" containerName="istio-proxy"
Sep 17 11:15:00.156510 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.156494    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-026f86-29827395-z5hdk"
Sep 17 11:15:00.157673 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.157657    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:15:00.163090 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.163069    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29827395-wxbtr"]
Sep 17 11:15:00.164712 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.164691    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827395-wxbtr"
Sep 17 11:15:00.170268 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.170247    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-026f86-29827395-z5hdk"]
Sep 17 11:15:00.173027 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.173006    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-026f86-29827395-z5hdk"
Sep 17 11:15:00.460924 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:15:00.460866    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0706d2a4_9a6e_4970_983b_b992ee72bce8.slice/crio-71c77f3e08949a95d13819c569c36b495cc3d1ac15dd9cdf5a84e48e93e3e8a0 WatchSource:0}: Error finding container 71c77f3e08949a95d13819c569c36b495cc3d1ac15dd9cdf5a84e48e93e3e8a0: Status 404 returned error can't find the container with id 71c77f3e08949a95d13819c569c36b495cc3d1ac15dd9cdf5a84e48e93e3e8a0
Sep 17 11:15:00.461433 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.461413    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29827395-wxbtr"]
Sep 17 11:15:00.476630 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:00.476600    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-026f86-29827395-z5hdk"]
Sep 17 11:15:00.482407 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:15:00.482383    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod985defe8_ac78_4a5b_9ce9_a9a26b4e5665.slice/crio-09399360a01d0f6f2fda082ec10fe12ed3bb75b3b6276e93c6ff7309d8b579c3 WatchSource:0}: Error finding container 09399360a01d0f6f2fda082ec10fe12ed3bb75b3b6276e93c6ff7309d8b579c3: Status 404 returned error can't find the container with id 09399360a01d0f6f2fda082ec10fe12ed3bb75b3b6276e93c6ff7309d8b579c3
Sep 17 11:15:01.142840 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:01.142810    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d42fcda-751a-4f67-a43d-b63c1fe19dd7" path="/var/lib/kubelet/pods/4d42fcda-751a-4f67-a43d-b63c1fe19dd7/volumes"
Sep 17 11:15:01.262115 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:01.262080    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827395-wxbtr" event={"ID":"0706d2a4-9a6e-4970-983b-b992ee72bce8","Type":"ContainerStarted","Data":"7d6ce86e0d87d0e3f4823fb1146d24e585f3805ca0fec06fc4cf8d44004acfb2"}
Sep 17 11:15:01.262556 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:01.262145    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827395-wxbtr" event={"ID":"0706d2a4-9a6e-4970-983b-b992ee72bce8","Type":"ContainerStarted","Data":"71c77f3e08949a95d13819c569c36b495cc3d1ac15dd9cdf5a84e48e93e3e8a0"}
Sep 17 11:15:01.263703 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:01.263681    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-026f86-29827395-z5hdk" event={"ID":"985defe8-ac78-4a5b-9ce9-a9a26b4e5665","Type":"ContainerStarted","Data":"49b3a7f73968440cbd5625ff6743bf3c30eda3a91b2835d48000430238d7a68c"}
Sep 17 11:15:01.263828 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:01.263728    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-026f86-29827395-z5hdk" event={"ID":"985defe8-ac78-4a5b-9ce9-a9a26b4e5665","Type":"ContainerStarted","Data":"09399360a01d0f6f2fda082ec10fe12ed3bb75b3b6276e93c6ff7309d8b579c3"}
Sep 17 11:15:01.293785 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:01.291812    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827395-wxbtr" podStartSLOduration=1.29179195 podStartE2EDuration="1.29179195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:15:01.286914655 +0000 UTC m=+1296.603277803" watchObservedRunningTime="2026-09-17 11:15:01.29179195 +0000 UTC m=+1296.608155105"
Sep 17 11:15:01.310241 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:01.310184    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-026f86-29827395-z5hdk" podStartSLOduration=1.310164058 podStartE2EDuration="1.310164058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:15:01.309091707 +0000 UTC m=+1296.625454860" watchObservedRunningTime="2026-09-17 11:15:01.310164058 +0000 UTC m=+1296.626527216"
Sep 17 11:15:02.271214 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:02.271180    2564 generic.go:356] "Generic (PLEG): container finished" podID="0706d2a4-9a6e-4970-983b-b992ee72bce8" containerID="7d6ce86e0d87d0e3f4823fb1146d24e585f3805ca0fec06fc4cf8d44004acfb2" exitCode=0
Sep 17 11:15:02.271648 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:02.271262    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827395-wxbtr" event={"ID":"0706d2a4-9a6e-4970-983b-b992ee72bce8","Type":"ContainerDied","Data":"7d6ce86e0d87d0e3f4823fb1146d24e585f3805ca0fec06fc4cf8d44004acfb2"}
Sep 17 11:15:02.272913 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:02.272869    2564 generic.go:356] "Generic (PLEG): container finished" podID="985defe8-ac78-4a5b-9ce9-a9a26b4e5665" containerID="49b3a7f73968440cbd5625ff6743bf3c30eda3a91b2835d48000430238d7a68c" exitCode=0
Sep 17 11:15:02.273031 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:02.272933    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-026f86-29827395-z5hdk" event={"ID":"985defe8-ac78-4a5b-9ce9-a9a26b4e5665","Type":"ContainerDied","Data":"49b3a7f73968440cbd5625ff6743bf3c30eda3a91b2835d48000430238d7a68c"}
Sep 17 11:15:03.435720 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:03.435698    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-026f86-29827395-z5hdk"
Sep 17 11:15:03.465095 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:03.464960    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827395-wxbtr"
Sep 17 11:15:04.286796 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:04.286769    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827395-wxbtr"
Sep 17 11:15:04.287020 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:04.286768    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827395-wxbtr" event={"ID":"0706d2a4-9a6e-4970-983b-b992ee72bce8","Type":"ContainerDied","Data":"71c77f3e08949a95d13819c569c36b495cc3d1ac15dd9cdf5a84e48e93e3e8a0"}
Sep 17 11:15:04.287020 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:04.286931    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71c77f3e08949a95d13819c569c36b495cc3d1ac15dd9cdf5a84e48e93e3e8a0"
Sep 17 11:15:04.288120 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:04.288103    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-026f86-29827395-z5hdk"
Sep 17 11:15:04.288120 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:04.288112    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-026f86-29827395-z5hdk" event={"ID":"985defe8-ac78-4a5b-9ce9-a9a26b4e5665","Type":"ContainerDied","Data":"09399360a01d0f6f2fda082ec10fe12ed3bb75b3b6276e93c6ff7309d8b579c3"}
Sep 17 11:15:04.288259 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:04.288152    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09399360a01d0f6f2fda082ec10fe12ed3bb75b3b6276e93c6ff7309d8b579c3"
Sep 17 11:15:08.301315 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.301279    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"]
Sep 17 11:15:08.301686 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.301658    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0706d2a4-9a6e-4970-983b-b992ee72bce8" containerName="cleanup"
Sep 17 11:15:08.301686 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.301670    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0706d2a4-9a6e-4970-983b-b992ee72bce8" containerName="cleanup"
Sep 17 11:15:08.301686 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.301679    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="985defe8-ac78-4a5b-9ce9-a9a26b4e5665" containerName="cleanup"
Sep 17 11:15:08.301686 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.301685    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="985defe8-ac78-4a5b-9ce9-a9a26b4e5665" containerName="cleanup"
Sep 17 11:15:08.301831 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.301735    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="985defe8-ac78-4a5b-9ce9-a9a26b4e5665" containerName="cleanup"
Sep 17 11:15:08.301831 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.301744    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0706d2a4-9a6e-4970-983b-b992ee72bce8" containerName="cleanup"
Sep 17 11:15:08.304875 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.304850    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.306962 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.306938    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-7751cd-openshift-default-dockercfg-4tz9m\""
Sep 17 11:15:08.326282 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.326260    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"]
Sep 17 11:15:08.440978 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.440942    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9884649-aa25-414b-9912-d2a01cffe74d-istiod-ca-cert\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.441150 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.440984    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9884649-aa25-414b-9912-d2a01cffe74d-istio-podinfo\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.441150 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.441086    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-credential-socket\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.441150 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.441116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-istio-data\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.441150 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.441143    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-istio-envoy\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.441313 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.441178    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-workload-certs\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.441313 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.441197    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dsmm\" (UniqueName: \"kubernetes.io/projected/b9884649-aa25-414b-9912-d2a01cffe74d-kube-api-access-2dsmm\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.441313 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.441230    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-workload-socket\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.441313 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.441275    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9884649-aa25-414b-9912-d2a01cffe74d-istio-token\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.542120 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.542083    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-workload-socket\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.542120 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.542121    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9884649-aa25-414b-9912-d2a01cffe74d-istio-token\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.542375 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.542155    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9884649-aa25-414b-9912-d2a01cffe74d-istiod-ca-cert\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.542375 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.542180    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9884649-aa25-414b-9912-d2a01cffe74d-istio-podinfo\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.542375 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.542226    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-credential-socket\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.542375 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.542244    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-istio-data\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.542375 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.542267    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-istio-envoy\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.542375 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.542303    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-workload-certs\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.542375 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.542328    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dsmm\" (UniqueName: \"kubernetes.io/projected/b9884649-aa25-414b-9912-d2a01cffe74d-kube-api-access-2dsmm\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.542757 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.542583    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-workload-socket\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.542757 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.542673    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-credential-socket\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.542757 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.542722    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-istio-data\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.542757 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.542750    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-workload-certs\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.543202 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.543182    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9884649-aa25-414b-9912-d2a01cffe74d-istiod-ca-cert\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.544701 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.544674    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-istio-envoy\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.544936 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.544876    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9884649-aa25-414b-9912-d2a01cffe74d-istio-podinfo\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.551697 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.551643    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9884649-aa25-414b-9912-d2a01cffe74d-istio-token\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.552129 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.552106    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dsmm\" (UniqueName: \"kubernetes.io/projected/b9884649-aa25-414b-9912-d2a01cffe74d-kube-api-access-2dsmm\") pod \"e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") " pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.617227 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.617203    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:08.896692 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.896668    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"]
Sep 17 11:15:08.898774 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:15:08.898747    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9884649_aa25_414b_9912_d2a01cffe74d.slice/crio-f79fa9e28245b39a8318fa8399d250cc7ae30ca17aa91621f6af3a3a9d1f2d61 WatchSource:0}: Error finding container f79fa9e28245b39a8318fa8399d250cc7ae30ca17aa91621f6af3a3a9d1f2d61: Status 404 returned error can't find the container with id f79fa9e28245b39a8318fa8399d250cc7ae30ca17aa91621f6af3a3a9d1f2d61
Sep 17 11:15:08.901056 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.901024    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:15:08.901146 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.901087    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:15:08.901146 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:08.901114    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:15:09.320449 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:09.320418    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj" event={"ID":"b9884649-aa25-414b-9912-d2a01cffe74d","Type":"ContainerStarted","Data":"ddd4eb6b43eadad0a5ddb2d1ba2fbb089558ad09b2ebc9cd4ada5d0a77fcbf30"}
Sep 17 11:15:09.320934 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:09.320477    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj" event={"ID":"b9884649-aa25-414b-9912-d2a01cffe74d","Type":"ContainerStarted","Data":"f79fa9e28245b39a8318fa8399d250cc7ae30ca17aa91621f6af3a3a9d1f2d61"}
Sep 17 11:15:09.363996 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:09.363945    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj" podStartSLOduration=1.363930035 podStartE2EDuration="1.363930035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:15:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:15:09.362161979 +0000 UTC m=+1304.678525135" watchObservedRunningTime="2026-09-17 11:15:09.363930035 +0000 UTC m=+1304.680293190"
Sep 17 11:15:09.617950 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:09.617842    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:09.623179 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:09.623156    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:10.327009 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:10.326971    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:10.327893 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:10.327875    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:13.306032 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.306000    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"]
Sep 17 11:15:13.309727 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.309706    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"
Sep 17 11:15:13.311963 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.311942    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-7751cd-dockercfg-rl2dn\""
Sep 17 11:15:13.312389 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.312373    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-7751cd\""
Sep 17 11:15:13.355756 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.355730    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"]
Sep 17 11:15:13.359133 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.359118    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"
Sep 17 11:15:13.378394 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.378369    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"]
Sep 17 11:15:13.381253 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.381234    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"]
Sep 17 11:15:13.491050 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.491016    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a357c44c-eb18-444f-b16f-e489371ab850-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt\" (UID: \"a357c44c-eb18-444f-b16f-e489371ab850\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"
Sep 17 11:15:13.491222 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.491066    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4nnr\" (UniqueName: \"kubernetes.io/projected/b8d2407c-c535-4c87-8bf7-6d7b0d91844a-kube-api-access-f4nnr\") pod \"payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42\" (UID: \"b8d2407c-c535-4c87-8bf7-6d7b0d91844a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"
Sep 17 11:15:13.491222 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.491190    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d2407c-c535-4c87-8bf7-6d7b0d91844a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42\" (UID: \"b8d2407c-c535-4c87-8bf7-6d7b0d91844a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"
Sep 17 11:15:13.491301 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.491241    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58rmk\" (UniqueName: \"kubernetes.io/projected/a357c44c-eb18-444f-b16f-e489371ab850-kube-api-access-58rmk\") pod \"payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt\" (UID: \"a357c44c-eb18-444f-b16f-e489371ab850\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"
Sep 17 11:15:13.592619 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.592538    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58rmk\" (UniqueName: \"kubernetes.io/projected/a357c44c-eb18-444f-b16f-e489371ab850-kube-api-access-58rmk\") pod \"payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt\" (UID: \"a357c44c-eb18-444f-b16f-e489371ab850\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"
Sep 17 11:15:13.592619 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.592591    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a357c44c-eb18-444f-b16f-e489371ab850-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt\" (UID: \"a357c44c-eb18-444f-b16f-e489371ab850\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"
Sep 17 11:15:13.592619 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.592620    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4nnr\" (UniqueName: \"kubernetes.io/projected/b8d2407c-c535-4c87-8bf7-6d7b0d91844a-kube-api-access-f4nnr\") pod \"payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42\" (UID: \"b8d2407c-c535-4c87-8bf7-6d7b0d91844a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"
Sep 17 11:15:13.592927 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.592671    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d2407c-c535-4c87-8bf7-6d7b0d91844a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42\" (UID: \"b8d2407c-c535-4c87-8bf7-6d7b0d91844a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"
Sep 17 11:15:13.593377 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.593357    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a357c44c-eb18-444f-b16f-e489371ab850-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt\" (UID: \"a357c44c-eb18-444f-b16f-e489371ab850\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"
Sep 17 11:15:13.593427 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.593378    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d2407c-c535-4c87-8bf7-6d7b0d91844a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42\" (UID: \"b8d2407c-c535-4c87-8bf7-6d7b0d91844a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"
Sep 17 11:15:13.617794 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.617769    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58rmk\" (UniqueName: \"kubernetes.io/projected/a357c44c-eb18-444f-b16f-e489371ab850-kube-api-access-58rmk\") pod \"payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt\" (UID: \"a357c44c-eb18-444f-b16f-e489371ab850\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"
Sep 17 11:15:13.620893 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.620864    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"
Sep 17 11:15:13.621762 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.621739    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4nnr\" (UniqueName: \"kubernetes.io/projected/b8d2407c-c535-4c87-8bf7-6d7b0d91844a-kube-api-access-f4nnr\") pod \"payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42\" (UID: \"b8d2407c-c535-4c87-8bf7-6d7b0d91844a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"
Sep 17 11:15:13.669197 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.669073    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"
Sep 17 11:15:13.907046 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.906973    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"]
Sep 17 11:15:13.909762 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:15:13.909735    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda357c44c_eb18_444f_b16f_e489371ab850.slice/crio-ea3c28ad493b94f02563abf274ad45a53ff5395ca30d5e4b8dedcdcf43dadefe WatchSource:0}: Error finding container ea3c28ad493b94f02563abf274ad45a53ff5395ca30d5e4b8dedcdcf43dadefe: Status 404 returned error can't find the container with id ea3c28ad493b94f02563abf274ad45a53ff5395ca30d5e4b8dedcdcf43dadefe
Sep 17 11:15:13.958138 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:13.958113    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"]
Sep 17 11:15:13.960937 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:15:13.960876    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8d2407c_c535_4c87_8bf7_6d7b0d91844a.slice/crio-7958d691fb184f52a5ebc827f42f43f0571b694088f6c417cbbd802988d7691a WatchSource:0}: Error finding container 7958d691fb184f52a5ebc827f42f43f0571b694088f6c417cbbd802988d7691a: Status 404 returned error can't find the container with id 7958d691fb184f52a5ebc827f42f43f0571b694088f6c417cbbd802988d7691a
Sep 17 11:15:14.353092 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:14.353054    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42" event={"ID":"b8d2407c-c535-4c87-8bf7-6d7b0d91844a","Type":"ContainerStarted","Data":"8863f37276d8c37a2bb73d93734a1479e3e6d69a83df5f419cbbc785267f899b"}
Sep 17 11:15:14.353516 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:14.353106    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42" event={"ID":"b8d2407c-c535-4c87-8bf7-6d7b0d91844a","Type":"ContainerStarted","Data":"7958d691fb184f52a5ebc827f42f43f0571b694088f6c417cbbd802988d7691a"}
Sep 17 11:15:14.353516 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:14.353196    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"
Sep 17 11:15:14.354794 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:14.354770    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt" event={"ID":"a357c44c-eb18-444f-b16f-e489371ab850","Type":"ContainerStarted","Data":"5d0f799b0e4154f19df2d92c170ebd8a77655ca8baec6b5f16a9c4b4b1ef052e"}
Sep 17 11:15:14.354927 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:14.354810    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt" event={"ID":"a357c44c-eb18-444f-b16f-e489371ab850","Type":"ContainerStarted","Data":"ea3c28ad493b94f02563abf274ad45a53ff5395ca30d5e4b8dedcdcf43dadefe"}
Sep 17 11:15:14.354927 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:14.354836    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"
Sep 17 11:15:14.403260 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:14.403204    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42" podStartSLOduration=1.403187121 podStartE2EDuration="1.403187121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:15:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:15:14.402350094 +0000 UTC m=+1309.718713251" watchObservedRunningTime="2026-09-17 11:15:14.403187121 +0000 UTC m=+1309.719550276"
Sep 17 11:15:19.137063 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:19.137025    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:15:25.363479 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:25.363427    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"
Sep 17 11:15:25.363860 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:25.363493    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"
Sep 17 11:15:25.383397 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:25.383347    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt" podStartSLOduration=12.383334108 podStartE2EDuration="12.383334108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:15:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:15:14.467848506 +0000 UTC m=+1309.784211661" watchObservedRunningTime="2026-09-17 11:15:25.383334108 +0000 UTC m=+1320.699697264"
Sep 17 11:15:26.370123 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:26.370072    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 17 11:15:26.370624 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:26.370144    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj" podUID="b9884649-aa25-414b-9912-d2a01cffe74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 17 11:15:28.456606 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:28.456521    2564 generic.go:356] "Generic (PLEG): container finished" podID="b9884649-aa25-414b-9912-d2a01cffe74d" containerID="ddd4eb6b43eadad0a5ddb2d1ba2fbb089558ad09b2ebc9cd4ada5d0a77fcbf30" exitCode=0
Sep 17 11:15:28.456606 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:28.456580    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj" event={"ID":"b9884649-aa25-414b-9912-d2a01cffe74d","Type":"ContainerDied","Data":"ddd4eb6b43eadad0a5ddb2d1ba2fbb089558ad09b2ebc9cd4ada5d0a77fcbf30"}
Sep 17 11:15:28.457056 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:28.456954    2564 scope.go:117] "RemoveContainer" containerID="ddd4eb6b43eadad0a5ddb2d1ba2fbb089558ad09b2ebc9cd4ada5d0a77fcbf30"
Sep 17 11:15:28.458029 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:28.458000    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:15:28.458108 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:28.458068    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:15:28.458108 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:28.458103    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:15:29.464147 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:29.464116    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj" event={"ID":"b9884649-aa25-414b-9912-d2a01cffe74d","Type":"ContainerStarted","Data":"c6ecd7d1057a4833fa205dc6ec0dbc8050c72ce29ea0f19641310ccccba88ad7"}
Sep 17 11:15:29.617616 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:29.617567    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:29.617880 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:29.617856    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 17 11:15:29.618014 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:29.617940    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj" podUID="b9884649-aa25-414b-9912-d2a01cffe74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 17 11:15:30.617569 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:30.617529    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 17 11:15:30.617952 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:30.617584    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj" podUID="b9884649-aa25-414b-9912-d2a01cffe74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 17 11:15:31.618431 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:31.618399    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 17 11:15:31.618817 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:31.618450    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj" podUID="b9884649-aa25-414b-9912-d2a01cffe74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 17 11:15:32.618341 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:32.618307    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 17 11:15:32.618521 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:32.618364    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj" podUID="b9884649-aa25-414b-9912-d2a01cffe74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 17 11:15:33.618474 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:33.618433    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 17 11:15:33.618635 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:33.618493    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj" podUID="b9884649-aa25-414b-9912-d2a01cffe74d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 17 11:15:34.692954 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:34.692924    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:34.693405 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:34.693306    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:34.694195 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:34.694174    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:15:43.676703 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:43.676670    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"]
Sep 17 11:15:43.679994 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:43.679977    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"
Sep 17 11:15:43.684728 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:43.684705    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-947f83-dockercfg-pncg2\""
Sep 17 11:15:43.685111 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:43.685090    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-947f83\""
Sep 17 11:15:43.705275 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:43.705243    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"]
Sep 17 11:15:43.766498 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:43.766466    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cfd6030-7451-40c5-984b-745fb276a505-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw\" (UID: \"5cfd6030-7451-40c5-984b-745fb276a505\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"
Sep 17 11:15:43.766685 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:43.766526    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6nkh\" (UniqueName: \"kubernetes.io/projected/5cfd6030-7451-40c5-984b-745fb276a505-kube-api-access-s6nkh\") pod \"payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw\" (UID: \"5cfd6030-7451-40c5-984b-745fb276a505\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"
Sep 17 11:15:43.867803 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:43.867760    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6nkh\" (UniqueName: \"kubernetes.io/projected/5cfd6030-7451-40c5-984b-745fb276a505-kube-api-access-s6nkh\") pod \"payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw\" (UID: \"5cfd6030-7451-40c5-984b-745fb276a505\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"
Sep 17 11:15:43.868104 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:43.867856    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cfd6030-7451-40c5-984b-745fb276a505-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw\" (UID: \"5cfd6030-7451-40c5-984b-745fb276a505\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"
Sep 17 11:15:43.868574 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:43.868551    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cfd6030-7451-40c5-984b-745fb276a505-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw\" (UID: \"5cfd6030-7451-40c5-984b-745fb276a505\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"
Sep 17 11:15:43.876322 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:43.876298    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6nkh\" (UniqueName: \"kubernetes.io/projected/5cfd6030-7451-40c5-984b-745fb276a505-kube-api-access-s6nkh\") pod \"payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw\" (UID: \"5cfd6030-7451-40c5-984b-745fb276a505\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"
Sep 17 11:15:43.990404 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:43.990378    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"
Sep 17 11:15:44.270549 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:44.270524    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"]
Sep 17 11:15:44.271470 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:15:44.271448    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cfd6030_7451_40c5_984b_745fb276a505.slice/crio-beebe8f689a6a315c4196011913108d409ecb17a169ca1ef84b04eb815fdbb0a WatchSource:0}: Error finding container beebe8f689a6a315c4196011913108d409ecb17a169ca1ef84b04eb815fdbb0a: Status 404 returned error can't find the container with id beebe8f689a6a315c4196011913108d409ecb17a169ca1ef84b04eb815fdbb0a
Sep 17 11:15:44.562026 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:44.561930    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw" event={"ID":"5cfd6030-7451-40c5-984b-745fb276a505","Type":"ContainerStarted","Data":"27d816c324a4f6796e2e8f62a76a2665a884c27ba77c9b939d69fbe786ce4df3"}
Sep 17 11:15:44.562026 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:44.561996    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw" event={"ID":"5cfd6030-7451-40c5-984b-745fb276a505","Type":"ContainerStarted","Data":"beebe8f689a6a315c4196011913108d409ecb17a169ca1ef84b04eb815fdbb0a"}
Sep 17 11:15:44.562261 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:44.562061    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"
Sep 17 11:15:44.598935 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:44.598862    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw" podStartSLOduration=1.598842512 podStartE2EDuration="1.598842512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:15:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:15:44.597388728 +0000 UTC m=+1339.913751885" watchObservedRunningTime="2026-09-17 11:15:44.598842512 +0000 UTC m=+1339.915205672"
Sep 17 11:15:55.570309 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:55.570239    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"
Sep 17 11:15:59.137211 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:15:59.137177    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:16:14.086878 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.086831    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"]
Sep 17 11:16:14.090722 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.090702    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.092932 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.092874    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d727624f-openshift-default-dockercfg-qr46w\""
Sep 17 11:16:14.101874 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.101820    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"]
Sep 17 11:16:14.147802 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.147773    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-envoy\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.148016 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.147825    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-credential-socket\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.148016 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.147864    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-workload-socket\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.148016 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.147921    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-token\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.148150 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.148018    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-podinfo\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.148150 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.148052    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-data\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.148150 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.148076    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2739541-8f5b-4618-9f39-822ebdafe9b2-istiod-ca-cert\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.148150 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.148092    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-workload-certs\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.148150 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.148110    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf4nn\" (UniqueName: \"kubernetes.io/projected/d2739541-8f5b-4618-9f39-822ebdafe9b2-kube-api-access-wf4nn\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.249529 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.249492    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-data\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.249529 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.249535    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2739541-8f5b-4618-9f39-822ebdafe9b2-istiod-ca-cert\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.249786 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.249553    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-workload-certs\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.249786 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.249569    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf4nn\" (UniqueName: \"kubernetes.io/projected/d2739541-8f5b-4618-9f39-822ebdafe9b2-kube-api-access-wf4nn\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.249786 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.249612    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-envoy\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.249786 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.249651    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-credential-socket\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.249786 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.249675    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-workload-socket\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.249786 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.249700    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-token\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.250155 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.249952    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-data\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.250155 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.249954    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-podinfo\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.250155 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.250087    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-workload-certs\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.250155 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.250118    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-workload-socket\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.250375 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.250210    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-credential-socket\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.250437 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.250416    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2739541-8f5b-4618-9f39-822ebdafe9b2-istiod-ca-cert\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.252345 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.252324    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-envoy\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.252456 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.252415    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-podinfo\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.257214 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.257191    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-token\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.257353 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.257333    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf4nn\" (UniqueName: \"kubernetes.io/projected/d2739541-8f5b-4618-9f39-822ebdafe9b2-kube-api-access-wf4nn\") pod \"e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") " pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.403113 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.403032    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:14.686589 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.686548    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"]
Sep 17 11:16:14.688457 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:16:14.688429    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2739541_8f5b_4618_9f39_822ebdafe9b2.slice/crio-ad45a48a39f5e3d6c178c5fd30bd749279d3369915f2f1e2f4f709583dea1009 WatchSource:0}: Error finding container ad45a48a39f5e3d6c178c5fd30bd749279d3369915f2f1e2f4f709583dea1009: Status 404 returned error can't find the container with id ad45a48a39f5e3d6c178c5fd30bd749279d3369915f2f1e2f4f709583dea1009
Sep 17 11:16:14.691403 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.691363    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:16:14.691482 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.691450    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:16:14.691521 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.691494    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:16:14.751326 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:14.751293    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl" event={"ID":"d2739541-8f5b-4618-9f39-822ebdafe9b2","Type":"ContainerStarted","Data":"ad45a48a39f5e3d6c178c5fd30bd749279d3369915f2f1e2f4f709583dea1009"}
Sep 17 11:16:15.759390 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:15.759353    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl" event={"ID":"d2739541-8f5b-4618-9f39-822ebdafe9b2","Type":"ContainerStarted","Data":"13117f8f346de7f64013cd3060c850a7d2b8cebc5abf8ab1397a000d96e717b9"}
Sep 17 11:16:15.787520 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:15.787466    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl" podStartSLOduration=1.78745165 podStartE2EDuration="1.78745165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:16:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:16:15.785356648 +0000 UTC m=+1371.101719816" watchObservedRunningTime="2026-09-17 11:16:15.78745165 +0000 UTC m=+1371.103814805"
Sep 17 11:16:16.403752 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:16.403716    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:16.409721 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:16.409692    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:16.765248 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:16.765216    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:16.766354 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:16.766338    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:25.323934 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:25.323881    2564 scope.go:117] "RemoveContainer" containerID="0ffe32164ad59d78e3fea25ac07d73b8bdb481d62ceacb2055e3f16ea91a1f5d"
Sep 17 11:16:40.137933 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:40.137869    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:16:41.930223 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:41.930193    2564 generic.go:356] "Generic (PLEG): container finished" podID="d2739541-8f5b-4618-9f39-822ebdafe9b2" containerID="13117f8f346de7f64013cd3060c850a7d2b8cebc5abf8ab1397a000d96e717b9" exitCode=0
Sep 17 11:16:41.930589 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:41.930240    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl" event={"ID":"d2739541-8f5b-4618-9f39-822ebdafe9b2","Type":"ContainerDied","Data":"13117f8f346de7f64013cd3060c850a7d2b8cebc5abf8ab1397a000d96e717b9"}
Sep 17 11:16:41.930713 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:41.930697    2564 scope.go:117] "RemoveContainer" containerID="13117f8f346de7f64013cd3060c850a7d2b8cebc5abf8ab1397a000d96e717b9"
Sep 17 11:16:41.931974 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:41.931943    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:16:41.932052 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:41.932036    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:16:41.932120 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:41.932076    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:16:42.938618 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:42.938582    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl" event={"ID":"d2739541-8f5b-4618-9f39-822ebdafe9b2","Type":"ContainerStarted","Data":"044b3e834fd0969652ab2fe674cd365788c3d6853482e429a02c51cb2c4f9d64"}
Sep 17 11:16:43.403533 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:43.403485    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:43.405275 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:43.405237    2564 patch_prober.go:28] interesting pod/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 17 11:16:43.405425 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:43.405308    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl" podUID="d2739541-8f5b-4618-9f39-822ebdafe9b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 17 11:16:44.404448 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:44.403982    2564 patch_prober.go:28] interesting pod/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 17 11:16:44.404448 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:44.404046    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl" podUID="d2739541-8f5b-4618-9f39-822ebdafe9b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 17 11:16:45.404516 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:45.404048    2564 patch_prober.go:28] interesting pod/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 17 11:16:45.404516 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:45.404114    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl" podUID="d2739541-8f5b-4618-9f39-822ebdafe9b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 17 11:16:46.404095 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:46.404051    2564 patch_prober.go:28] interesting pod/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 17 11:16:46.404421 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:46.404389    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl" podUID="d2739541-8f5b-4618-9f39-822ebdafe9b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 17 11:16:46.766166 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:46.766118    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:47.404424 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:47.404382    2564 patch_prober.go:28] interesting pod/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 17 11:16:47.404652 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:47.404474    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl" podUID="d2739541-8f5b-4618-9f39-822ebdafe9b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 17 11:16:48.407639 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:48.407609    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:16:48.408640 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:16:48.408619    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:17:04.137320 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:04.137287    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:17:22.931382 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:22.931354    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7"]
Sep 17 11:17:22.935037 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:22.935020    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7"
Sep 17 11:17:22.937404 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:22.937378    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:17:22.937724 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:22.937709    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:17:22.947148 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:22.947128    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7"]
Sep 17 11:17:23.078645 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:23.078610    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0228b145-99d8-4faa-8571-b6b3c4b71a4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d727624f-wbjg7\" (UID: \"0228b145-99d8-4faa-8571-b6b3c4b71a4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7"
Sep 17 11:17:23.179503 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:23.179472    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0228b145-99d8-4faa-8571-b6b3c4b71a4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d727624f-wbjg7\" (UID: \"0228b145-99d8-4faa-8571-b6b3c4b71a4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7"
Sep 17 11:17:23.180116 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:23.180097    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0228b145-99d8-4faa-8571-b6b3c4b71a4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d727624f-wbjg7\" (UID: \"0228b145-99d8-4faa-8571-b6b3c4b71a4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7"
Sep 17 11:17:23.246595 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:23.246571    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7"
Sep 17 11:17:23.520940 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:23.520913    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7"]
Sep 17 11:17:23.522986 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:17:23.522962    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0228b145_99d8_4faa_8571_b6b3c4b71a4d.slice/crio-f708c11cb7fc9c1ad06ad7f9cbe0e46a466d24bc9cc5ac50f6c2bce27ed38d9b WatchSource:0}: Error finding container f708c11cb7fc9c1ad06ad7f9cbe0e46a466d24bc9cc5ac50f6c2bce27ed38d9b: Status 404 returned error can't find the container with id f708c11cb7fc9c1ad06ad7f9cbe0e46a466d24bc9cc5ac50f6c2bce27ed38d9b
Sep 17 11:17:24.189786 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:24.189755    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7" event={"ID":"0228b145-99d8-4faa-8571-b6b3c4b71a4d","Type":"ContainerStarted","Data":"4c7e9f6bbeeb8ebc6e6009982b2b9c7a151fcd66e05369bf821ea9262664c1b8"}
Sep 17 11:17:24.190174 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:24.189806    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7" event={"ID":"0228b145-99d8-4faa-8571-b6b3c4b71a4d","Type":"ContainerStarted","Data":"f708c11cb7fc9c1ad06ad7f9cbe0e46a466d24bc9cc5ac50f6c2bce27ed38d9b"}
Sep 17 11:17:24.206413 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:24.206361    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7" podStartSLOduration=2.206346961 podStartE2EDuration="2.206346961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:17:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:17:24.204046033 +0000 UTC m=+1439.520409187" watchObservedRunningTime="2026-09-17 11:17:24.206346961 +0000 UTC m=+1439.522710115"
Sep 17 11:17:25.197781 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:25.197750    2564 generic.go:356] "Generic (PLEG): container finished" podID="0228b145-99d8-4faa-8571-b6b3c4b71a4d" containerID="4c7e9f6bbeeb8ebc6e6009982b2b9c7a151fcd66e05369bf821ea9262664c1b8" exitCode=0
Sep 17 11:17:25.198185 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:25.197831    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7" event={"ID":"0228b145-99d8-4faa-8571-b6b3c4b71a4d","Type":"ContainerDied","Data":"4c7e9f6bbeeb8ebc6e6009982b2b9c7a151fcd66e05369bf821ea9262664c1b8"}
Sep 17 11:17:25.346615 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:25.346585    2564 scope.go:117] "RemoveContainer" containerID="f22c20aa92fe979e422ba6caad7fadde2af04d2fd80df95a6d9b7c4289cdc4db"
Sep 17 11:17:25.355612 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:25.355594    2564 scope.go:117] "RemoveContainer" containerID="02b2b60d90cc2db75f444d2b759d3c79213ba977812a7ffd340ec0522f864321"
Sep 17 11:17:26.342786 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:26.342761    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7"
Sep 17 11:17:26.509071 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:26.509043    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0228b145-99d8-4faa-8571-b6b3c4b71a4d-maas-api-service-ca\") pod \"0228b145-99d8-4faa-8571-b6b3c4b71a4d\" (UID: \"0228b145-99d8-4faa-8571-b6b3c4b71a4d\") "
Sep 17 11:17:26.509389 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:26.509367    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0228b145-99d8-4faa-8571-b6b3c4b71a4d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0228b145-99d8-4faa-8571-b6b3c4b71a4d" (UID: "0228b145-99d8-4faa-8571-b6b3c4b71a4d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:17:26.610465 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:26.610434    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0228b145-99d8-4faa-8571-b6b3c4b71a4d-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:17:27.209816 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:27.209786    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7"
Sep 17 11:17:27.209984 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:27.209836    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7" event={"ID":"0228b145-99d8-4faa-8571-b6b3c4b71a4d","Type":"ContainerDied","Data":"f708c11cb7fc9c1ad06ad7f9cbe0e46a466d24bc9cc5ac50f6c2bce27ed38d9b"}
Sep 17 11:17:27.209984 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:27.209881    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f708c11cb7fc9c1ad06ad7f9cbe0e46a466d24bc9cc5ac50f6c2bce27ed38d9b"
Sep 17 11:17:38.999441 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:38.999409    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7"]
Sep 17 11:17:39.005588 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:39.005563    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d727624f-wbjg7"]
Sep 17 11:17:39.154040 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:39.154004    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0228b145-99d8-4faa-8571-b6b3c4b71a4d" path="/var/lib/kubelet/pods/0228b145-99d8-4faa-8571-b6b3c4b71a4d/volumes"
Sep 17 11:17:48.137682 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:17:48.137651    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:18:12.136789 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:12.136755    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:18:25.381675 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:25.381646    2564 scope.go:117] "RemoveContainer" containerID="1b53531b22721fbc78635b1607fbaf8270d2947d31ba7738818d01f5932fd6de"
Sep 17 11:18:25.391345 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:25.391328    2564 scope.go:117] "RemoveContainer" containerID="5fc949bb598ea57517c0e9aa656258151d941a43f4d44896c6de7fb45b1150da"
Sep 17 11:18:25.399873 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:25.399851    2564 scope.go:117] "RemoveContainer" containerID="3dbfc0be789e4fc23d7aaa70ba2a1908778101ce0bab1c784fab50de15c77e59"
Sep 17 11:18:31.297579 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:31.297540    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"]
Sep 17 11:18:31.299602 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:31.298403    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl" podUID="d2739541-8f5b-4618-9f39-822ebdafe9b2" containerName="istio-proxy" containerID="cri-o://044b3e834fd0969652ab2fe674cd365788c3d6853482e429a02c51cb2c4f9d64" gracePeriod=30
Sep 17 11:18:33.410416 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:33.410374    2564 patch_prober.go:28] interesting pod/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 17 11:18:33.410792 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:33.410432    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl" podUID="d2739541-8f5b-4618-9f39-822ebdafe9b2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 11:18:36.553825 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.553802    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:18:36.632336 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.632254    2564 generic.go:356] "Generic (PLEG): container finished" podID="d2739541-8f5b-4618-9f39-822ebdafe9b2" containerID="044b3e834fd0969652ab2fe674cd365788c3d6853482e429a02c51cb2c4f9d64" exitCode=0
Sep 17 11:18:36.632336 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.632301    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"
Sep 17 11:18:36.632537 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.632336    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl" event={"ID":"d2739541-8f5b-4618-9f39-822ebdafe9b2","Type":"ContainerDied","Data":"044b3e834fd0969652ab2fe674cd365788c3d6853482e429a02c51cb2c4f9d64"}
Sep 17 11:18:36.632537 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.632391    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl" event={"ID":"d2739541-8f5b-4618-9f39-822ebdafe9b2","Type":"ContainerDied","Data":"ad45a48a39f5e3d6c178c5fd30bd749279d3369915f2f1e2f4f709583dea1009"}
Sep 17 11:18:36.632537 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.632413    2564 scope.go:117] "RemoveContainer" containerID="044b3e834fd0969652ab2fe674cd365788c3d6853482e429a02c51cb2c4f9d64"
Sep 17 11:18:36.642039 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.642023    2564 scope.go:117] "RemoveContainer" containerID="13117f8f346de7f64013cd3060c850a7d2b8cebc5abf8ab1397a000d96e717b9"
Sep 17 11:18:36.650952 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.650932    2564 scope.go:117] "RemoveContainer" containerID="044b3e834fd0969652ab2fe674cd365788c3d6853482e429a02c51cb2c4f9d64"
Sep 17 11:18:36.651234 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:18:36.651215    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"044b3e834fd0969652ab2fe674cd365788c3d6853482e429a02c51cb2c4f9d64\": container with ID starting with 044b3e834fd0969652ab2fe674cd365788c3d6853482e429a02c51cb2c4f9d64 not found: ID does not exist" containerID="044b3e834fd0969652ab2fe674cd365788c3d6853482e429a02c51cb2c4f9d64"
Sep 17 11:18:36.651284 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.651244    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"044b3e834fd0969652ab2fe674cd365788c3d6853482e429a02c51cb2c4f9d64"} err="failed to get container status \"044b3e834fd0969652ab2fe674cd365788c3d6853482e429a02c51cb2c4f9d64\": rpc error: code = NotFound desc = could not find container \"044b3e834fd0969652ab2fe674cd365788c3d6853482e429a02c51cb2c4f9d64\": container with ID starting with 044b3e834fd0969652ab2fe674cd365788c3d6853482e429a02c51cb2c4f9d64 not found: ID does not exist"
Sep 17 11:18:36.651284 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.651263    2564 scope.go:117] "RemoveContainer" containerID="13117f8f346de7f64013cd3060c850a7d2b8cebc5abf8ab1397a000d96e717b9"
Sep 17 11:18:36.651535 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:18:36.651515    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13117f8f346de7f64013cd3060c850a7d2b8cebc5abf8ab1397a000d96e717b9\": container with ID starting with 13117f8f346de7f64013cd3060c850a7d2b8cebc5abf8ab1397a000d96e717b9 not found: ID does not exist" containerID="13117f8f346de7f64013cd3060c850a7d2b8cebc5abf8ab1397a000d96e717b9"
Sep 17 11:18:36.651592 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.651544    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13117f8f346de7f64013cd3060c850a7d2b8cebc5abf8ab1397a000d96e717b9"} err="failed to get container status \"13117f8f346de7f64013cd3060c850a7d2b8cebc5abf8ab1397a000d96e717b9\": rpc error: code = NotFound desc = could not find container \"13117f8f346de7f64013cd3060c850a7d2b8cebc5abf8ab1397a000d96e717b9\": container with ID starting with 13117f8f346de7f64013cd3060c850a7d2b8cebc5abf8ab1397a000d96e717b9 not found: ID does not exist"
Sep 17 11:18:36.654856 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.654839    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wf4nn\" (UniqueName: \"kubernetes.io/projected/d2739541-8f5b-4618-9f39-822ebdafe9b2-kube-api-access-wf4nn\") pod \"d2739541-8f5b-4618-9f39-822ebdafe9b2\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") "
Sep 17 11:18:36.654945 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.654876    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-workload-certs\") pod \"d2739541-8f5b-4618-9f39-822ebdafe9b2\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") "
Sep 17 11:18:36.654945 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.654933    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-envoy\") pod \"d2739541-8f5b-4618-9f39-822ebdafe9b2\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") "
Sep 17 11:18:36.655042 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.654966    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-workload-socket\") pod \"d2739541-8f5b-4618-9f39-822ebdafe9b2\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") "
Sep 17 11:18:36.655042 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.654995    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2739541-8f5b-4618-9f39-822ebdafe9b2-istiod-ca-cert\") pod \"d2739541-8f5b-4618-9f39-822ebdafe9b2\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") "
Sep 17 11:18:36.655138 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.655043    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-podinfo\") pod \"d2739541-8f5b-4618-9f39-822ebdafe9b2\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") "
Sep 17 11:18:36.655138 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.655087    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-credential-socket\") pod \"d2739541-8f5b-4618-9f39-822ebdafe9b2\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") "
Sep 17 11:18:36.655138 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.655113    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-token\") pod \"d2739541-8f5b-4618-9f39-822ebdafe9b2\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") "
Sep 17 11:18:36.655295 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.655155    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-data\") pod \"d2739541-8f5b-4618-9f39-822ebdafe9b2\" (UID: \"d2739541-8f5b-4618-9f39-822ebdafe9b2\") "
Sep 17 11:18:36.655295 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.655209    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d2739541-8f5b-4618-9f39-822ebdafe9b2" (UID: "d2739541-8f5b-4618-9f39-822ebdafe9b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:18:36.655295 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.655259    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d2739541-8f5b-4618-9f39-822ebdafe9b2" (UID: "d2739541-8f5b-4618-9f39-822ebdafe9b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:18:36.655441 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.655396    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d2739541-8f5b-4618-9f39-822ebdafe9b2" (UID: "d2739541-8f5b-4618-9f39-822ebdafe9b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:18:36.655492 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.655465    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "d2739541-8f5b-4618-9f39-822ebdafe9b2" (UID: "d2739541-8f5b-4618-9f39-822ebdafe9b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:18:36.655561 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.655539    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-workload-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:18:36.655618 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.655567    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-credential-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:18:36.655618 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.655584    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-data\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:18:36.655618 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.655599    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-workload-certs\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:18:36.655738 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.655653    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2739541-8f5b-4618-9f39-822ebdafe9b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d2739541-8f5b-4618-9f39-822ebdafe9b2" (UID: "d2739541-8f5b-4618-9f39-822ebdafe9b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:18:36.657098 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.657071    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2739541-8f5b-4618-9f39-822ebdafe9b2-kube-api-access-wf4nn" (OuterVolumeSpecName: "kube-api-access-wf4nn") pod "d2739541-8f5b-4618-9f39-822ebdafe9b2" (UID: "d2739541-8f5b-4618-9f39-822ebdafe9b2"). InnerVolumeSpecName "kube-api-access-wf4nn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:18:36.657216 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.657197    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d2739541-8f5b-4618-9f39-822ebdafe9b2" (UID: "d2739541-8f5b-4618-9f39-822ebdafe9b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:18:36.657396 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.657372    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "d2739541-8f5b-4618-9f39-822ebdafe9b2" (UID: "d2739541-8f5b-4618-9f39-822ebdafe9b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:18:36.657556 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.657539    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d2739541-8f5b-4618-9f39-822ebdafe9b2" (UID: "d2739541-8f5b-4618-9f39-822ebdafe9b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 11:18:36.756249 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.756211    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2739541-8f5b-4618-9f39-822ebdafe9b2-istiod-ca-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:18:36.756249 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.756240    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-podinfo\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:18:36.756249 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.756249    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-token\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:18:36.756249 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.756259    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wf4nn\" (UniqueName: \"kubernetes.io/projected/d2739541-8f5b-4618-9f39-822ebdafe9b2-kube-api-access-wf4nn\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:18:36.756514 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.756268    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2739541-8f5b-4618-9f39-822ebdafe9b2-istio-envoy\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:18:36.955491 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.955464    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"]
Sep 17 11:18:36.959778 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:36.959753    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d727624f-openshift-default-5ff84dbd76-qdtcl"]
Sep 17 11:18:37.150436 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:18:37.144569    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2739541-8f5b-4618-9f39-822ebdafe9b2" path="/var/lib/kubelet/pods/d2739541-8f5b-4618-9f39-822ebdafe9b2/volumes"
Sep 17 11:19:13.137340 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:13.137303    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:19:25.429599 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:25.429566    2564 scope.go:117] "RemoveContainer" containerID="0494857c37cf526d091bb9902cb9dcc9abd4eb1cfc94c294a482dc76373f6e60"
Sep 17 11:19:30.136796 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:30.136765    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:19:34.293769 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.293731    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"]
Sep 17 11:19:34.294209 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.294195    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2739541-8f5b-4618-9f39-822ebdafe9b2" containerName="istio-proxy"
Sep 17 11:19:34.294254 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.294210    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2739541-8f5b-4618-9f39-822ebdafe9b2" containerName="istio-proxy"
Sep 17 11:19:34.294254 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.294219    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2739541-8f5b-4618-9f39-822ebdafe9b2" containerName="istio-proxy"
Sep 17 11:19:34.294254 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.294226    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2739541-8f5b-4618-9f39-822ebdafe9b2" containerName="istio-proxy"
Sep 17 11:19:34.294254 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.294239    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0228b145-99d8-4faa-8571-b6b3c4b71a4d" containerName="revoke-keys"
Sep 17 11:19:34.294254 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.294244    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0228b145-99d8-4faa-8571-b6b3c4b71a4d" containerName="revoke-keys"
Sep 17 11:19:34.294407 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.294304    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2739541-8f5b-4618-9f39-822ebdafe9b2" containerName="istio-proxy"
Sep 17 11:19:34.294407 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.294313    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2739541-8f5b-4618-9f39-822ebdafe9b2" containerName="istio-proxy"
Sep 17 11:19:34.294407 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.294320    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0228b145-99d8-4faa-8571-b6b3c4b71a4d" containerName="revoke-keys"
Sep 17 11:19:34.299043 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.299027    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"
Sep 17 11:19:34.305363 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.305301    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-c6abf3-dockercfg-dg2xg\""
Sep 17 11:19:34.305929 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.305623    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-c6abf3\""
Sep 17 11:19:34.335522 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.335497    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"]
Sep 17 11:19:34.388656 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.388624    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj4wb\" (UniqueName: \"kubernetes.io/projected/15f5c1e7-b68b-46da-92f1-092b14b781d1-kube-api-access-hj4wb\") pod \"payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths\" (UID: \"15f5c1e7-b68b-46da-92f1-092b14b781d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"
Sep 17 11:19:34.388803 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.388700    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15f5c1e7-b68b-46da-92f1-092b14b781d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths\" (UID: \"15f5c1e7-b68b-46da-92f1-092b14b781d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"
Sep 17 11:19:34.489981 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.489948    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj4wb\" (UniqueName: \"kubernetes.io/projected/15f5c1e7-b68b-46da-92f1-092b14b781d1-kube-api-access-hj4wb\") pod \"payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths\" (UID: \"15f5c1e7-b68b-46da-92f1-092b14b781d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"
Sep 17 11:19:34.490128 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.489995    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15f5c1e7-b68b-46da-92f1-092b14b781d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths\" (UID: \"15f5c1e7-b68b-46da-92f1-092b14b781d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"
Sep 17 11:19:34.490604 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.490587    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15f5c1e7-b68b-46da-92f1-092b14b781d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths\" (UID: \"15f5c1e7-b68b-46da-92f1-092b14b781d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"
Sep 17 11:19:34.500213 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.500181    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj4wb\" (UniqueName: \"kubernetes.io/projected/15f5c1e7-b68b-46da-92f1-092b14b781d1-kube-api-access-hj4wb\") pod \"payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths\" (UID: \"15f5c1e7-b68b-46da-92f1-092b14b781d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"
Sep 17 11:19:34.609993 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.609931    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"
Sep 17 11:19:34.893245 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.893220    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"]
Sep 17 11:19:34.894611 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:19:34.894584    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15f5c1e7_b68b_46da_92f1_092b14b781d1.slice/crio-4197ef780cae9aa00f1d0770b8cb10531f0b60130a8b2270cc679ec232350e13 WatchSource:0}: Error finding container 4197ef780cae9aa00f1d0770b8cb10531f0b60130a8b2270cc679ec232350e13: Status 404 returned error can't find the container with id 4197ef780cae9aa00f1d0770b8cb10531f0b60130a8b2270cc679ec232350e13
Sep 17 11:19:34.970234 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.970204    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths" event={"ID":"15f5c1e7-b68b-46da-92f1-092b14b781d1","Type":"ContainerStarted","Data":"5cb6e4d3c6883238bb28d0a1ef3cf5a54c83118a04acf4f6d800cf5403ff6e46"}
Sep 17 11:19:34.970349 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.970263    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths" event={"ID":"15f5c1e7-b68b-46da-92f1-092b14b781d1","Type":"ContainerStarted","Data":"4197ef780cae9aa00f1d0770b8cb10531f0b60130a8b2270cc679ec232350e13"}
Sep 17 11:19:34.970428 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.970374    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"
Sep 17 11:19:34.987147 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:34.987104    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths" podStartSLOduration=0.987089119 podStartE2EDuration="987.089119ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:19:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:19:34.985603991 +0000 UTC m=+1570.301967142" watchObservedRunningTime="2026-09-17 11:19:34.987089119 +0000 UTC m=+1570.303452274"
Sep 17 11:19:45.979126 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:19:45.979093    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"
Sep 17 11:20:01.255365 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.255336    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"]
Sep 17 11:20:01.259111 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.259094    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.261838 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.261821    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-9f97d7-openshift-default-dockercfg-f7lnl\""
Sep 17 11:20:01.279872 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.279841    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"]
Sep 17 11:20:01.329249 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.329220    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-istio-data\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.329249 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.329250    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f647c211-a46d-4ccb-8021-39a19888e795-istiod-ca-cert\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.329433 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.329291    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f647c211-a46d-4ccb-8021-39a19888e795-istio-token\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.329433 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.329307    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp6xr\" (UniqueName: \"kubernetes.io/projected/f647c211-a46d-4ccb-8021-39a19888e795-kube-api-access-bp6xr\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.329433 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.329333    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-istio-envoy\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.329433 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.329367    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-workload-certs\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.329433 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.329380    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f647c211-a46d-4ccb-8021-39a19888e795-istio-podinfo\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.329433 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.329404    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-credential-socket\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.329672 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.329447    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-workload-socket\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.430820 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.430785    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f647c211-a46d-4ccb-8021-39a19888e795-istio-token\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.430820 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.430822    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp6xr\" (UniqueName: \"kubernetes.io/projected/f647c211-a46d-4ccb-8021-39a19888e795-kube-api-access-bp6xr\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.431102 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.430846    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-istio-envoy\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.431102 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.430893    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-workload-certs\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.431102 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.430947    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f647c211-a46d-4ccb-8021-39a19888e795-istio-podinfo\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.431102 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.430988    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-credential-socket\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.431102 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.431033    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-workload-socket\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.431413 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.431122    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-istio-data\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.431413 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.431146    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f647c211-a46d-4ccb-8021-39a19888e795-istiod-ca-cert\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.431516 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.431439    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-workload-certs\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.431516 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.431470    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-credential-socket\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.431621 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.431571    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-workload-socket\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.431771 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.431626    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-istio-data\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.432039 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.432014    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f647c211-a46d-4ccb-8021-39a19888e795-istiod-ca-cert\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.433485 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.433459    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-istio-envoy\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.433689 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.433670    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f647c211-a46d-4ccb-8021-39a19888e795-istio-podinfo\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.444782 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.444754    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f647c211-a46d-4ccb-8021-39a19888e795-istio-token\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.444920 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.444782    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp6xr\" (UniqueName: \"kubernetes.io/projected/f647c211-a46d-4ccb-8021-39a19888e795-kube-api-access-bp6xr\") pod \"e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") " pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.571746 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.571662    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:01.861442 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.861416    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"]
Sep 17 11:20:01.863004 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:20:01.862979    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf647c211_a46d_4ccb_8021_39a19888e795.slice/crio-d332be80e07351dd79512bab60c21a71955ebbf8754c7663229ce737c6c484b1 WatchSource:0}: Error finding container d332be80e07351dd79512bab60c21a71955ebbf8754c7663229ce737c6c484b1: Status 404 returned error can't find the container with id d332be80e07351dd79512bab60c21a71955ebbf8754c7663229ce737c6c484b1
Sep 17 11:20:01.865128 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.865095    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:20:01.865225 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.865159    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:20:01.865225 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:01.865196    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:20:02.132644 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:02.132548    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x" event={"ID":"f647c211-a46d-4ccb-8021-39a19888e795","Type":"ContainerStarted","Data":"637f71d572671fefa4f5c950fd160f562c6f4e8e8d21516ce12bf501da245bfc"}
Sep 17 11:20:02.132644 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:02.132605    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x" event={"ID":"f647c211-a46d-4ccb-8021-39a19888e795","Type":"ContainerStarted","Data":"d332be80e07351dd79512bab60c21a71955ebbf8754c7663229ce737c6c484b1"}
Sep 17 11:20:02.155820 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:02.155757    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x" podStartSLOduration=1.155740741 podStartE2EDuration="1.155740741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:20:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:20:02.154170385 +0000 UTC m=+1597.470533534" watchObservedRunningTime="2026-09-17 11:20:02.155740741 +0000 UTC m=+1597.472103897"
Sep 17 11:20:02.572920 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:02.572855    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:02.578237 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:02.578211    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:03.144070 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:03.144044    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:03.144236 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:03.144111    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:15.823342 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:15.823312    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"]
Sep 17 11:20:15.827589 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:15.827567    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:15.830633 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:15.830612    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-31fca0-openshift-default-dockercfg-bkmkm\""
Sep 17 11:20:15.843758 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:15.843736    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"]
Sep 17 11:20:15.966724 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:15.966693    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-data\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:15.966882 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:15.966741    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-workload-certs\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:15.966882 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:15.966760    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-credential-socket\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:15.966882 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:15.966796    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-podinfo\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:15.966882 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:15.966826    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istiod-ca-cert\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:15.967051 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:15.966880    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rgqd\" (UniqueName: \"kubernetes.io/projected/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-kube-api-access-6rgqd\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:15.967051 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:15.966945    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-token\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:15.967051 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:15.966975    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-envoy\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:15.967144 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:15.967074    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-workload-socket\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.068066 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.068025    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-workload-certs\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.068284 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.068071    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-credential-socket\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.068284 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.068259    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-podinfo\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.068396 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.068312    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istiod-ca-cert\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.068396 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.068351    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rgqd\" (UniqueName: \"kubernetes.io/projected/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-kube-api-access-6rgqd\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.068396 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.068382    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-token\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.068541 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.068400    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-workload-certs\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.068541 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.068423    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-credential-socket\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.068541 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.068427    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-envoy\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.068695 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.068529    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-workload-socket\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.068695 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.068591    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-data\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.068967 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.068881    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-workload-socket\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.068967 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.068954    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-data\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.069131 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.069071    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istiod-ca-cert\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.071396 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.071377    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-envoy\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.071546 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.071531    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-podinfo\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.078948 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.078866    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rgqd\" (UniqueName: \"kubernetes.io/projected/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-kube-api-access-6rgqd\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.079969 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.079939    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-token\") pod \"e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") " pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.139064 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.139043    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:16.420765 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.420743    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"]
Sep 17 11:20:16.421960 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:20:16.421935    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a9c5855_f6e1_48c7_a4ea_cf703069a72f.slice/crio-16c29375c89db6746d125564ecb8dc7fd6862df1910ad0824e80fd957bbe8285 WatchSource:0}: Error finding container 16c29375c89db6746d125564ecb8dc7fd6862df1910ad0824e80fd957bbe8285: Status 404 returned error can't find the container with id 16c29375c89db6746d125564ecb8dc7fd6862df1910ad0824e80fd957bbe8285
Sep 17 11:20:16.424763 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.424728    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:20:16.424848 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.424817    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:20:16.424959 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:16.424916    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:20:17.234700 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:17.234661    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng" event={"ID":"8a9c5855-f6e1-48c7-a4ea-cf703069a72f","Type":"ContainerStarted","Data":"face60727f7637d3dfc38df7697ba04d289ecfe8749ad3516f031629b0ed89bf"}
Sep 17 11:20:17.235177 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:17.234723    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng" event={"ID":"8a9c5855-f6e1-48c7-a4ea-cf703069a72f","Type":"ContainerStarted","Data":"16c29375c89db6746d125564ecb8dc7fd6862df1910ad0824e80fd957bbe8285"}
Sep 17 11:20:17.262796 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:17.262747    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng" podStartSLOduration=2.262732228 podStartE2EDuration="2.262732228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:20:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:20:17.259199933 +0000 UTC m=+1612.575563087" watchObservedRunningTime="2026-09-17 11:20:17.262732228 +0000 UTC m=+1612.579095440"
Sep 17 11:20:18.139647 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:18.139613    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:18.145036 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:18.145014    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:18.240956 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:18.240881    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:18.241824 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:18.241807    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:24.929853 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:24.929820    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"]
Sep 17 11:20:24.933797 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:24.933776    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"
Sep 17 11:20:24.937638 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:24.937616    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-31fca0\""
Sep 17 11:20:24.939246 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:24.939224    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-31fca0-dockercfg-lws7g\""
Sep 17 11:20:24.961143 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:24.961120    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"]
Sep 17 11:20:24.967007 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:24.966977    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sb5z\" (UniqueName: \"kubernetes.io/projected/81613d36-c3b8-4a8a-889b-a3a663aa9e1e-kube-api-access-5sb5z\") pod \"payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n\" (UID: \"81613d36-c3b8-4a8a-889b-a3a663aa9e1e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"
Sep 17 11:20:24.967153 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:24.967116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81613d36-c3b8-4a8a-889b-a3a663aa9e1e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n\" (UID: \"81613d36-c3b8-4a8a-889b-a3a663aa9e1e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"
Sep 17 11:20:25.067874 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:25.067842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sb5z\" (UniqueName: \"kubernetes.io/projected/81613d36-c3b8-4a8a-889b-a3a663aa9e1e-kube-api-access-5sb5z\") pod \"payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n\" (UID: \"81613d36-c3b8-4a8a-889b-a3a663aa9e1e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"
Sep 17 11:20:25.068098 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:25.067938    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81613d36-c3b8-4a8a-889b-a3a663aa9e1e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n\" (UID: \"81613d36-c3b8-4a8a-889b-a3a663aa9e1e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"
Sep 17 11:20:25.070510 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:25.070487    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-31fca0\""
Sep 17 11:20:25.076745 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:25.076726    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sb5z\" (UniqueName: \"kubernetes.io/projected/81613d36-c3b8-4a8a-889b-a3a663aa9e1e-kube-api-access-5sb5z\") pod \"payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n\" (UID: \"81613d36-c3b8-4a8a-889b-a3a663aa9e1e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"
Sep 17 11:20:25.079333 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:25.079309    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81613d36-c3b8-4a8a-889b-a3a663aa9e1e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n\" (UID: \"81613d36-c3b8-4a8a-889b-a3a663aa9e1e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"
Sep 17 11:20:25.247331 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:25.247307    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-31fca0-dockercfg-lws7g\""
Sep 17 11:20:25.256309 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:25.256291    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"
Sep 17 11:20:25.457292 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:25.457255    2564 scope.go:117] "RemoveContainer" containerID="6faadcf155286e9eaba9fb8832bb5a1f51f98ec529dbc51b747c46f6d7cb2bc9"
Sep 17 11:20:25.540636 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:25.540607    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"]
Sep 17 11:20:25.547758 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:20:25.547729    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81613d36_c3b8_4a8a_889b_a3a663aa9e1e.slice/crio-b2b460a8e4bab2cb01712994c87ce06152a098a2f3b9c7cb9dd53c67642ea526 WatchSource:0}: Error finding container b2b460a8e4bab2cb01712994c87ce06152a098a2f3b9c7cb9dd53c67642ea526: Status 404 returned error can't find the container with id b2b460a8e4bab2cb01712994c87ce06152a098a2f3b9c7cb9dd53c67642ea526
Sep 17 11:20:26.293054 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:26.293021    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n" event={"ID":"81613d36-c3b8-4a8a-889b-a3a663aa9e1e","Type":"ContainerStarted","Data":"572b0cd15ede155cb0bf1bf97e07f6878f13e29d2d5a885db4cd532b22f43514"}
Sep 17 11:20:26.293440 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:26.293073    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n" event={"ID":"81613d36-c3b8-4a8a-889b-a3a663aa9e1e","Type":"ContainerStarted","Data":"b2b460a8e4bab2cb01712994c87ce06152a098a2f3b9c7cb9dd53c67642ea526"}
Sep 17 11:20:26.293440 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:26.293110    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"
Sep 17 11:20:26.311124 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:26.311060    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n" podStartSLOduration=2.311040675 podStartE2EDuration="2.311040675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:20:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:20:26.309589259 +0000 UTC m=+1621.625952414" watchObservedRunningTime="2026-09-17 11:20:26.311040675 +0000 UTC m=+1621.627403829"
Sep 17 11:20:26.987208 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:26.987178    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc"]
Sep 17 11:20:26.989956 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:26.989939    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc"
Sep 17 11:20:26.992294 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:26.992270    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:20:26.992417 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:26.992358    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:20:27.000140 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:27.000116    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc"]
Sep 17 11:20:27.184788 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:27.184751    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcaccc3d-243c-4ea5-ba50-ac3785d8f787-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc\" (UID: \"dcaccc3d-243c-4ea5-ba50-ac3785d8f787\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc"
Sep 17 11:20:27.286209 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:27.286130    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcaccc3d-243c-4ea5-ba50-ac3785d8f787-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc\" (UID: \"dcaccc3d-243c-4ea5-ba50-ac3785d8f787\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc"
Sep 17 11:20:27.286714 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:27.286694    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcaccc3d-243c-4ea5-ba50-ac3785d8f787-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc\" (UID: \"dcaccc3d-243c-4ea5-ba50-ac3785d8f787\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc"
Sep 17 11:20:27.300126 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:27.300098    2564 generic.go:356] "Generic (PLEG): container finished" podID="f647c211-a46d-4ccb-8021-39a19888e795" containerID="637f71d572671fefa4f5c950fd160f562c6f4e8e8d21516ce12bf501da245bfc" exitCode=0
Sep 17 11:20:27.300469 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:27.300173    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x" event={"ID":"f647c211-a46d-4ccb-8021-39a19888e795","Type":"ContainerDied","Data":"637f71d572671fefa4f5c950fd160f562c6f4e8e8d21516ce12bf501da245bfc"}
Sep 17 11:20:27.300675 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:27.300662    2564 scope.go:117] "RemoveContainer" containerID="637f71d572671fefa4f5c950fd160f562c6f4e8e8d21516ce12bf501da245bfc"
Sep 17 11:20:27.300987 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:27.300973    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc"
Sep 17 11:20:27.301807 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:27.301777    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:20:27.301878 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:27.301848    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:20:27.301942 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:27.301880    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:20:27.582711 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:27.582671    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc"]
Sep 17 11:20:28.313323 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:28.313209    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc" event={"ID":"dcaccc3d-243c-4ea5-ba50-ac3785d8f787","Type":"ContainerStarted","Data":"985f9ac3cd138fb923aab9798dfd9154b85525595c27dfc0dc969cfb5869a986"}
Sep 17 11:20:28.313323 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:28.313288    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc" event={"ID":"dcaccc3d-243c-4ea5-ba50-ac3785d8f787","Type":"ContainerStarted","Data":"891404d231cdda47d4dbfe7098962c5ec63a8e70168465b25ccbf86a4b7e1dd9"}
Sep 17 11:20:28.316273 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:28.316248    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x" event={"ID":"f647c211-a46d-4ccb-8021-39a19888e795","Type":"ContainerStarted","Data":"abc4d1a07a30179925104ac09180347dc35a290683ae4dc1f1df1868fa600ba8"}
Sep 17 11:20:28.333586 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:28.333541    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc" podStartSLOduration=2.333526116 podStartE2EDuration="2.333526116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:20:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:20:28.329859841 +0000 UTC m=+1623.646222996" watchObservedRunningTime="2026-09-17 11:20:28.333526116 +0000 UTC m=+1623.649889271"
Sep 17 11:20:28.572582 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:28.572505    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:28.573546 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:28.573520    2564 patch_prober.go:28] interesting pod/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 17 11:20:28.573663 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:28.573577    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x" podUID="f647c211-a46d-4ccb-8021-39a19888e795" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 17 11:20:29.325571 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:29.325532    2564 generic.go:356] "Generic (PLEG): container finished" podID="dcaccc3d-243c-4ea5-ba50-ac3785d8f787" containerID="985f9ac3cd138fb923aab9798dfd9154b85525595c27dfc0dc969cfb5869a986" exitCode=0
Sep 17 11:20:29.325986 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:29.325670    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc" event={"ID":"dcaccc3d-243c-4ea5-ba50-ac3785d8f787","Type":"ContainerDied","Data":"985f9ac3cd138fb923aab9798dfd9154b85525595c27dfc0dc969cfb5869a986"}
Sep 17 11:20:29.572420 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:29.572381    2564 patch_prober.go:28] interesting pod/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 17 11:20:29.572593 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:29.572438    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x" podUID="f647c211-a46d-4ccb-8021-39a19888e795" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 17 11:20:30.137242 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:30.137206    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:20:30.526307 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:30.526280    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc"
Sep 17 11:20:30.572337 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:30.572297    2564 patch_prober.go:28] interesting pod/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 17 11:20:30.572520 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:30.572360    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x" podUID="f647c211-a46d-4ccb-8021-39a19888e795" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 17 11:20:30.623594 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:30.623557    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcaccc3d-243c-4ea5-ba50-ac3785d8f787-maas-api-service-ca\") pod \"dcaccc3d-243c-4ea5-ba50-ac3785d8f787\" (UID: \"dcaccc3d-243c-4ea5-ba50-ac3785d8f787\") "
Sep 17 11:20:30.623928 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:30.623876    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcaccc3d-243c-4ea5-ba50-ac3785d8f787-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dcaccc3d-243c-4ea5-ba50-ac3785d8f787" (UID: "dcaccc3d-243c-4ea5-ba50-ac3785d8f787"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:20:30.725171 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:30.725072    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcaccc3d-243c-4ea5-ba50-ac3785d8f787-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:20:31.342194 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:31.342163    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc"
Sep 17 11:20:31.342370 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:31.342194    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc" event={"ID":"dcaccc3d-243c-4ea5-ba50-ac3785d8f787","Type":"ContainerDied","Data":"891404d231cdda47d4dbfe7098962c5ec63a8e70168465b25ccbf86a4b7e1dd9"}
Sep 17 11:20:31.342370 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:31.342236    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="891404d231cdda47d4dbfe7098962c5ec63a8e70168465b25ccbf86a4b7e1dd9"
Sep 17 11:20:31.572678 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:31.572641    2564 patch_prober.go:28] interesting pod/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 17 11:20:31.573127 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:31.572698    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x" podUID="f647c211-a46d-4ccb-8021-39a19888e795" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 17 11:20:32.572768 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:32.572718    2564 patch_prober.go:28] interesting pod/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 17 11:20:32.573235 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:32.572790    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x" podUID="f647c211-a46d-4ccb-8021-39a19888e795" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 17 11:20:33.143945 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:33.143917    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:33.576184 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:33.576154    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:33.577289 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:33.577268    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:20:37.303471 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:37.303439    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"
Sep 17 11:20:42.860418 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:42.860387    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"]
Sep 17 11:20:42.861268 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:42.861228    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths" podUID="15f5c1e7-b68b-46da-92f1-092b14b781d1" containerName="payload-pre-processing" containerID="cri-o://5cb6e4d3c6883238bb28d0a1ef3cf5a54c83118a04acf4f6d800cf5403ff6e46" gracePeriod=30
Sep 17 11:20:43.145396 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.145369    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc"]
Sep 17 11:20:43.145625 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.145608    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"
Sep 17 11:20:43.154143 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.154110    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c6abf3-d5qfc"]
Sep 17 11:20:43.264862 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.264831    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15f5c1e7-b68b-46da-92f1-092b14b781d1-plugins-config-volume\") pod \"15f5c1e7-b68b-46da-92f1-092b14b781d1\" (UID: \"15f5c1e7-b68b-46da-92f1-092b14b781d1\") "
Sep 17 11:20:43.265104 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.264978    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hj4wb\" (UniqueName: \"kubernetes.io/projected/15f5c1e7-b68b-46da-92f1-092b14b781d1-kube-api-access-hj4wb\") pod \"15f5c1e7-b68b-46da-92f1-092b14b781d1\" (UID: \"15f5c1e7-b68b-46da-92f1-092b14b781d1\") "
Sep 17 11:20:43.265366 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.265334    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15f5c1e7-b68b-46da-92f1-092b14b781d1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "15f5c1e7-b68b-46da-92f1-092b14b781d1" (UID: "15f5c1e7-b68b-46da-92f1-092b14b781d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:20:43.267192 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.267169    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15f5c1e7-b68b-46da-92f1-092b14b781d1-kube-api-access-hj4wb" (OuterVolumeSpecName: "kube-api-access-hj4wb") pod "15f5c1e7-b68b-46da-92f1-092b14b781d1" (UID: "15f5c1e7-b68b-46da-92f1-092b14b781d1"). InnerVolumeSpecName "kube-api-access-hj4wb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:20:43.366325 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.366287    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15f5c1e7-b68b-46da-92f1-092b14b781d1-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:20:43.366325 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.366320    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj4wb\" (UniqueName: \"kubernetes.io/projected/15f5c1e7-b68b-46da-92f1-092b14b781d1-kube-api-access-hj4wb\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:20:43.428593 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.428509    2564 generic.go:356] "Generic (PLEG): container finished" podID="15f5c1e7-b68b-46da-92f1-092b14b781d1" containerID="5cb6e4d3c6883238bb28d0a1ef3cf5a54c83118a04acf4f6d800cf5403ff6e46" exitCode=0
Sep 17 11:20:43.428593 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.428567    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"
Sep 17 11:20:43.428804 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.428596    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths" event={"ID":"15f5c1e7-b68b-46da-92f1-092b14b781d1","Type":"ContainerDied","Data":"5cb6e4d3c6883238bb28d0a1ef3cf5a54c83118a04acf4f6d800cf5403ff6e46"}
Sep 17 11:20:43.428804 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.428664    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths" event={"ID":"15f5c1e7-b68b-46da-92f1-092b14b781d1","Type":"ContainerDied","Data":"4197ef780cae9aa00f1d0770b8cb10531f0b60130a8b2270cc679ec232350e13"}
Sep 17 11:20:43.428804 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.428693    2564 scope.go:117] "RemoveContainer" containerID="5cb6e4d3c6883238bb28d0a1ef3cf5a54c83118a04acf4f6d800cf5403ff6e46"
Sep 17 11:20:43.438231 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.438214    2564 scope.go:117] "RemoveContainer" containerID="5cb6e4d3c6883238bb28d0a1ef3cf5a54c83118a04acf4f6d800cf5403ff6e46"
Sep 17 11:20:43.438525 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:20:43.438501    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cb6e4d3c6883238bb28d0a1ef3cf5a54c83118a04acf4f6d800cf5403ff6e46\": container with ID starting with 5cb6e4d3c6883238bb28d0a1ef3cf5a54c83118a04acf4f6d800cf5403ff6e46 not found: ID does not exist" containerID="5cb6e4d3c6883238bb28d0a1ef3cf5a54c83118a04acf4f6d800cf5403ff6e46"
Sep 17 11:20:43.438603 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.438537    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cb6e4d3c6883238bb28d0a1ef3cf5a54c83118a04acf4f6d800cf5403ff6e46"} err="failed to get container status \"5cb6e4d3c6883238bb28d0a1ef3cf5a54c83118a04acf4f6d800cf5403ff6e46\": rpc error: code = NotFound desc = could not find container \"5cb6e4d3c6883238bb28d0a1ef3cf5a54c83118a04acf4f6d800cf5403ff6e46\": container with ID starting with 5cb6e4d3c6883238bb28d0a1ef3cf5a54c83118a04acf4f6d800cf5403ff6e46 not found: ID does not exist"
Sep 17 11:20:43.466537 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.466511    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"]
Sep 17 11:20:43.480996 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:43.480972    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c6abf3-786464d986-psths"]
Sep 17 11:20:44.436103 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:44.436008    2564 generic.go:356] "Generic (PLEG): container finished" podID="8a9c5855-f6e1-48c7-a4ea-cf703069a72f" containerID="face60727f7637d3dfc38df7697ba04d289ecfe8749ad3516f031629b0ed89bf" exitCode=0
Sep 17 11:20:44.436579 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:44.436089    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng" event={"ID":"8a9c5855-f6e1-48c7-a4ea-cf703069a72f","Type":"ContainerDied","Data":"face60727f7637d3dfc38df7697ba04d289ecfe8749ad3516f031629b0ed89bf"}
Sep 17 11:20:44.436579 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:44.436564    2564 scope.go:117] "RemoveContainer" containerID="face60727f7637d3dfc38df7697ba04d289ecfe8749ad3516f031629b0ed89bf"
Sep 17 11:20:44.437835 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:44.437809    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:20:44.437925 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:44.437876    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:20:44.437973 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:44.437932    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:20:45.143700 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:45.143671    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15f5c1e7-b68b-46da-92f1-092b14b781d1" path="/var/lib/kubelet/pods/15f5c1e7-b68b-46da-92f1-092b14b781d1/volumes"
Sep 17 11:20:45.144019 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:45.144005    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcaccc3d-243c-4ea5-ba50-ac3785d8f787" path="/var/lib/kubelet/pods/dcaccc3d-243c-4ea5-ba50-ac3785d8f787/volumes"
Sep 17 11:20:45.444854 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:45.444772    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng" event={"ID":"8a9c5855-f6e1-48c7-a4ea-cf703069a72f","Type":"ContainerStarted","Data":"da34249788cc6e548a193b61be5812d60d9f08f620d28b5262f7f64f9406092d"}
Sep 17 11:20:46.139449 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:46.139410    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:46.140812 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:46.140781    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 17 11:20:46.140996 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:46.140831    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng" podUID="8a9c5855-f6e1-48c7-a4ea-cf703069a72f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 17 11:20:47.139998 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:47.139959    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 17 11:20:47.140373 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:47.140026    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng" podUID="8a9c5855-f6e1-48c7-a4ea-cf703069a72f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 17 11:20:48.140309 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:48.140265    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 17 11:20:48.140780 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:48.140337    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng" podUID="8a9c5855-f6e1-48c7-a4ea-cf703069a72f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 17 11:20:48.241313 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:48.241278    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:49.139673 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:49.139634    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 17 11:20:49.140017 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:49.139695    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng" podUID="8a9c5855-f6e1-48c7-a4ea-cf703069a72f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 17 11:20:50.139614 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:50.139578    2564 patch_prober.go:28] interesting pod/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 17 11:20:50.140035 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:50.139629    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng" podUID="8a9c5855-f6e1-48c7-a4ea-cf703069a72f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 17 11:20:51.148562 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.148520    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:51.150486 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.150459    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:20:51.404114 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.404027    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b"]
Sep 17 11:20:51.404479 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.404465    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcaccc3d-243c-4ea5-ba50-ac3785d8f787" containerName="revoke-keys"
Sep 17 11:20:51.404534 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.404482    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcaccc3d-243c-4ea5-ba50-ac3785d8f787" containerName="revoke-keys"
Sep 17 11:20:51.404534 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.404502    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15f5c1e7-b68b-46da-92f1-092b14b781d1" containerName="payload-pre-processing"
Sep 17 11:20:51.404534 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.404508    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="15f5c1e7-b68b-46da-92f1-092b14b781d1" containerName="payload-pre-processing"
Sep 17 11:20:51.404658 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.404566    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcaccc3d-243c-4ea5-ba50-ac3785d8f787" containerName="revoke-keys"
Sep 17 11:20:51.404658 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.404578    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="15f5c1e7-b68b-46da-92f1-092b14b781d1" containerName="payload-pre-processing"
Sep 17 11:20:51.415166 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.415133    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b"
Sep 17 11:20:51.416093 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.416068    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b"]
Sep 17 11:20:51.419134 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.419112    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:20:51.419295 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.419270    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:20:51.555260 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.555230    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7765b9fa-b8dd-40c2-8cd2-96e6db65ec36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b\" (UID: \"7765b9fa-b8dd-40c2-8cd2-96e6db65ec36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b"
Sep 17 11:20:51.656716 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.656622    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7765b9fa-b8dd-40c2-8cd2-96e6db65ec36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b\" (UID: \"7765b9fa-b8dd-40c2-8cd2-96e6db65ec36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b"
Sep 17 11:20:51.657243 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.657222    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7765b9fa-b8dd-40c2-8cd2-96e6db65ec36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b\" (UID: \"7765b9fa-b8dd-40c2-8cd2-96e6db65ec36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b"
Sep 17 11:20:51.726996 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:51.726962    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b"
Sep 17 11:20:52.022439 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:52.022391    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b"]
Sep 17 11:20:52.026615 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:20:52.026568    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7765b9fa_b8dd_40c2_8cd2_96e6db65ec36.slice/crio-4bd982bd78d6749173a57feee8bebe75464f1f420fe61a05254e618623d59638 WatchSource:0}: Error finding container 4bd982bd78d6749173a57feee8bebe75464f1f420fe61a05254e618623d59638: Status 404 returned error can't find the container with id 4bd982bd78d6749173a57feee8bebe75464f1f420fe61a05254e618623d59638
Sep 17 11:20:52.490151 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:52.490119    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b" event={"ID":"7765b9fa-b8dd-40c2-8cd2-96e6db65ec36","Type":"ContainerStarted","Data":"12940c3d69f424fef7e40f5fdeb59f74944a5d87c2ebcc840c4bb829b2d4bccf"}
Sep 17 11:20:52.490551 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:52.490173    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b" event={"ID":"7765b9fa-b8dd-40c2-8cd2-96e6db65ec36","Type":"ContainerStarted","Data":"4bd982bd78d6749173a57feee8bebe75464f1f420fe61a05254e618623d59638"}
Sep 17 11:20:52.523473 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:52.523409    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b" podStartSLOduration=1.523393645 podStartE2EDuration="1.523393645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:20:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:20:52.5205568 +0000 UTC m=+1647.836919956" watchObservedRunningTime="2026-09-17 11:20:52.523393645 +0000 UTC m=+1647.839756844"
Sep 17 11:20:53.497774 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:53.497744    2564 generic.go:356] "Generic (PLEG): container finished" podID="7765b9fa-b8dd-40c2-8cd2-96e6db65ec36" containerID="12940c3d69f424fef7e40f5fdeb59f74944a5d87c2ebcc840c4bb829b2d4bccf" exitCode=0
Sep 17 11:20:53.498165 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:53.497825    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b" event={"ID":"7765b9fa-b8dd-40c2-8cd2-96e6db65ec36","Type":"ContainerDied","Data":"12940c3d69f424fef7e40f5fdeb59f74944a5d87c2ebcc840c4bb829b2d4bccf"}
Sep 17 11:20:54.644414 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:54.644390    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b"
Sep 17 11:20:54.789300 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:54.789221    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7765b9fa-b8dd-40c2-8cd2-96e6db65ec36-maas-api-service-ca\") pod \"7765b9fa-b8dd-40c2-8cd2-96e6db65ec36\" (UID: \"7765b9fa-b8dd-40c2-8cd2-96e6db65ec36\") "
Sep 17 11:20:54.789571 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:54.789547    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7765b9fa-b8dd-40c2-8cd2-96e6db65ec36-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7765b9fa-b8dd-40c2-8cd2-96e6db65ec36" (UID: "7765b9fa-b8dd-40c2-8cd2-96e6db65ec36"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:20:54.890389 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:54.890360    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7765b9fa-b8dd-40c2-8cd2-96e6db65ec36-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:20:55.511527 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:55.511500    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b"
Sep 17 11:20:55.511704 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:55.511533    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b" event={"ID":"7765b9fa-b8dd-40c2-8cd2-96e6db65ec36","Type":"ContainerDied","Data":"4bd982bd78d6749173a57feee8bebe75464f1f420fe61a05254e618623d59638"}
Sep 17 11:20:55.511704 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:20:55.511584    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4bd982bd78d6749173a57feee8bebe75464f1f420fe61a05254e618623d59638"
Sep 17 11:21:00.137549 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:00.137509    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:21:07.808273 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:07.808219    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b"]
Sep 17 11:21:07.811218 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:07.811194    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9f97d7-jp59b"]
Sep 17 11:21:09.146045 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:09.146007    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7765b9fa-b8dd-40c2-8cd2-96e6db65ec36" path="/var/lib/kubelet/pods/7765b9fa-b8dd-40c2-8cd2-96e6db65ec36/volumes"
Sep 17 11:21:14.949150 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:14.949120    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"]
Sep 17 11:21:14.949624 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:14.949343    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x" podUID="f647c211-a46d-4ccb-8021-39a19888e795" containerName="istio-proxy" containerID="cri-o://abc4d1a07a30179925104ac09180347dc35a290683ae4dc1f1df1868fa600ba8" gracePeriod=30
Sep 17 11:21:15.353144 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:15.353098    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq"]
Sep 17 11:21:15.353614 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:15.353595    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7765b9fa-b8dd-40c2-8cd2-96e6db65ec36" containerName="revoke-keys"
Sep 17 11:21:15.353710 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:15.353617    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7765b9fa-b8dd-40c2-8cd2-96e6db65ec36" containerName="revoke-keys"
Sep 17 11:21:15.353771 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:15.353747    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7765b9fa-b8dd-40c2-8cd2-96e6db65ec36" containerName="revoke-keys"
Sep 17 11:21:15.358061 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:15.358039    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq"
Sep 17 11:21:15.360219 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:15.360199    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:21:15.360325 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:15.360204    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:21:15.366050 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:15.366022    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq"]
Sep 17 11:21:15.487963 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:15.487926    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8356b49-dcb8-4495-8755-daec0f256512-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq\" (UID: \"c8356b49-dcb8-4495-8755-daec0f256512\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq"
Sep 17 11:21:15.589422 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:15.589382    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8356b49-dcb8-4495-8755-daec0f256512-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq\" (UID: \"c8356b49-dcb8-4495-8755-daec0f256512\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq"
Sep 17 11:21:15.590048 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:15.590028    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8356b49-dcb8-4495-8755-daec0f256512-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq\" (UID: \"c8356b49-dcb8-4495-8755-daec0f256512\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq"
Sep 17 11:21:15.671188 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:15.671085    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq"
Sep 17 11:21:15.948639 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:15.948609    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq"]
Sep 17 11:21:15.950855 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:21:15.950831    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8356b49_dcb8_4495_8755_daec0f256512.slice/crio-85fa483060c5718a534795c2b7df9a15f8b05d665ad378403b57288f8be6f412 WatchSource:0}: Error finding container 85fa483060c5718a534795c2b7df9a15f8b05d665ad378403b57288f8be6f412: Status 404 returned error can't find the container with id 85fa483060c5718a534795c2b7df9a15f8b05d665ad378403b57288f8be6f412
Sep 17 11:21:16.642737 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:16.642699    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq" event={"ID":"c8356b49-dcb8-4495-8755-daec0f256512","Type":"ContainerStarted","Data":"368a7e0b61d9c05bd2dc7aece91d8b25dced24993946fc6991128e25b44b7375"}
Sep 17 11:21:16.642964 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:16.642753    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq" event={"ID":"c8356b49-dcb8-4495-8755-daec0f256512","Type":"ContainerStarted","Data":"85fa483060c5718a534795c2b7df9a15f8b05d665ad378403b57288f8be6f412"}
Sep 17 11:21:16.659434 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:16.659384    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq" podStartSLOduration=1.6593654660000001 podStartE2EDuration="1.659365466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:21:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:21:16.657631979 +0000 UTC m=+1671.973995137" watchObservedRunningTime="2026-09-17 11:21:16.659365466 +0000 UTC m=+1671.975728621"
Sep 17 11:21:17.650759 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:17.650722    2564 generic.go:356] "Generic (PLEG): container finished" podID="c8356b49-dcb8-4495-8755-daec0f256512" containerID="368a7e0b61d9c05bd2dc7aece91d8b25dced24993946fc6991128e25b44b7375" exitCode=0
Sep 17 11:21:17.651182 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:17.650804    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq" event={"ID":"c8356b49-dcb8-4495-8755-daec0f256512","Type":"ContainerDied","Data":"368a7e0b61d9c05bd2dc7aece91d8b25dced24993946fc6991128e25b44b7375"}
Sep 17 11:21:18.579189 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:18.579151    2564 patch_prober.go:28] interesting pod/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 17 11:21:18.579361 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:18.579215    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x" podUID="f647c211-a46d-4ccb-8021-39a19888e795" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 11:21:18.796978 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:18.796946    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq"
Sep 17 11:21:18.920252 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:18.920160    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8356b49-dcb8-4495-8755-daec0f256512-maas-api-service-ca\") pod \"c8356b49-dcb8-4495-8755-daec0f256512\" (UID: \"c8356b49-dcb8-4495-8755-daec0f256512\") "
Sep 17 11:21:18.920529 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:18.920502    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8356b49-dcb8-4495-8755-daec0f256512-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c8356b49-dcb8-4495-8755-daec0f256512" (UID: "c8356b49-dcb8-4495-8755-daec0f256512"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:21:19.021823 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:19.021790    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8356b49-dcb8-4495-8755-daec0f256512-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:19.667556 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:19.667516    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq" event={"ID":"c8356b49-dcb8-4495-8755-daec0f256512","Type":"ContainerDied","Data":"85fa483060c5718a534795c2b7df9a15f8b05d665ad378403b57288f8be6f412"}
Sep 17 11:21:19.667556 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:19.667539    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq"
Sep 17 11:21:19.667771 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:19.667570    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85fa483060c5718a534795c2b7df9a15f8b05d665ad378403b57288f8be6f412"
Sep 17 11:21:20.185629 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.185604    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:21:20.333569 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.333540    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f647c211-a46d-4ccb-8021-39a19888e795-istio-token\") pod \"f647c211-a46d-4ccb-8021-39a19888e795\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") "
Sep 17 11:21:20.333751 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.333585    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-istio-data\") pod \"f647c211-a46d-4ccb-8021-39a19888e795\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") "
Sep 17 11:21:20.333751 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.333611    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-workload-certs\") pod \"f647c211-a46d-4ccb-8021-39a19888e795\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") "
Sep 17 11:21:20.333751 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.333657    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-workload-socket\") pod \"f647c211-a46d-4ccb-8021-39a19888e795\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") "
Sep 17 11:21:20.333751 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.333695    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f647c211-a46d-4ccb-8021-39a19888e795-istio-podinfo\") pod \"f647c211-a46d-4ccb-8021-39a19888e795\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") "
Sep 17 11:21:20.333751 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.333740    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f647c211-a46d-4ccb-8021-39a19888e795-istiod-ca-cert\") pod \"f647c211-a46d-4ccb-8021-39a19888e795\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") "
Sep 17 11:21:20.334057 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.333788    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bp6xr\" (UniqueName: \"kubernetes.io/projected/f647c211-a46d-4ccb-8021-39a19888e795-kube-api-access-bp6xr\") pod \"f647c211-a46d-4ccb-8021-39a19888e795\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") "
Sep 17 11:21:20.334057 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.333814    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-credential-socket\") pod \"f647c211-a46d-4ccb-8021-39a19888e795\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") "
Sep 17 11:21:20.334057 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.333864    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-istio-data" (OuterVolumeSpecName: "istio-data") pod "f647c211-a46d-4ccb-8021-39a19888e795" (UID: "f647c211-a46d-4ccb-8021-39a19888e795"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:21:20.334057 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.333873    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-istio-envoy\") pod \"f647c211-a46d-4ccb-8021-39a19888e795\" (UID: \"f647c211-a46d-4ccb-8021-39a19888e795\") "
Sep 17 11:21:20.334267 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.334050    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f647c211-a46d-4ccb-8021-39a19888e795" (UID: "f647c211-a46d-4ccb-8021-39a19888e795"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:21:20.334267 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.334122    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f647c211-a46d-4ccb-8021-39a19888e795" (UID: "f647c211-a46d-4ccb-8021-39a19888e795"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:21:20.334267 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.334177    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f647c211-a46d-4ccb-8021-39a19888e795-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f647c211-a46d-4ccb-8021-39a19888e795" (UID: "f647c211-a46d-4ccb-8021-39a19888e795"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:21:20.334593 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.334275    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-istio-data\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:20.334593 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.334295    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-workload-certs\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:20.334593 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.334309    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f647c211-a46d-4ccb-8021-39a19888e795-istiod-ca-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:20.334593 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.334322    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-credential-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:20.334820 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.334700    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f647c211-a46d-4ccb-8021-39a19888e795" (UID: "f647c211-a46d-4ccb-8021-39a19888e795"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:21:20.336100 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.336066    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f647c211-a46d-4ccb-8021-39a19888e795-kube-api-access-bp6xr" (OuterVolumeSpecName: "kube-api-access-bp6xr") pod "f647c211-a46d-4ccb-8021-39a19888e795" (UID: "f647c211-a46d-4ccb-8021-39a19888e795"). InnerVolumeSpecName "kube-api-access-bp6xr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:21:20.336100 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.336088    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f647c211-a46d-4ccb-8021-39a19888e795-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f647c211-a46d-4ccb-8021-39a19888e795" (UID: "f647c211-a46d-4ccb-8021-39a19888e795"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 11:21:20.336238 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.336131    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f647c211-a46d-4ccb-8021-39a19888e795-istio-token" (OuterVolumeSpecName: "istio-token") pod "f647c211-a46d-4ccb-8021-39a19888e795" (UID: "f647c211-a46d-4ccb-8021-39a19888e795"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:21:20.336238 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.336137    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f647c211-a46d-4ccb-8021-39a19888e795" (UID: "f647c211-a46d-4ccb-8021-39a19888e795"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:21:20.435338 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.435307    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f647c211-a46d-4ccb-8021-39a19888e795-istio-token\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:20.435338 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.435334    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-workload-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:20.435495 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.435348    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f647c211-a46d-4ccb-8021-39a19888e795-istio-podinfo\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:20.435495 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.435361    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp6xr\" (UniqueName: \"kubernetes.io/projected/f647c211-a46d-4ccb-8021-39a19888e795-kube-api-access-bp6xr\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:20.435495 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.435372    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f647c211-a46d-4ccb-8021-39a19888e795-istio-envoy\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:20.675832 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.675745    2564 generic.go:356] "Generic (PLEG): container finished" podID="f647c211-a46d-4ccb-8021-39a19888e795" containerID="abc4d1a07a30179925104ac09180347dc35a290683ae4dc1f1df1868fa600ba8" exitCode=0
Sep 17 11:21:20.675832 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.675804    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"
Sep 17 11:21:20.676062 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.675825    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x" event={"ID":"f647c211-a46d-4ccb-8021-39a19888e795","Type":"ContainerDied","Data":"abc4d1a07a30179925104ac09180347dc35a290683ae4dc1f1df1868fa600ba8"}
Sep 17 11:21:20.676062 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.675881    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x" event={"ID":"f647c211-a46d-4ccb-8021-39a19888e795","Type":"ContainerDied","Data":"d332be80e07351dd79512bab60c21a71955ebbf8754c7663229ce737c6c484b1"}
Sep 17 11:21:20.676062 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.675929    2564 scope.go:117] "RemoveContainer" containerID="abc4d1a07a30179925104ac09180347dc35a290683ae4dc1f1df1868fa600ba8"
Sep 17 11:21:20.686230 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.686212    2564 scope.go:117] "RemoveContainer" containerID="637f71d572671fefa4f5c950fd160f562c6f4e8e8d21516ce12bf501da245bfc"
Sep 17 11:21:20.695043 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.694980    2564 scope.go:117] "RemoveContainer" containerID="abc4d1a07a30179925104ac09180347dc35a290683ae4dc1f1df1868fa600ba8"
Sep 17 11:21:20.695552 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:21:20.695455    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abc4d1a07a30179925104ac09180347dc35a290683ae4dc1f1df1868fa600ba8\": container with ID starting with abc4d1a07a30179925104ac09180347dc35a290683ae4dc1f1df1868fa600ba8 not found: ID does not exist" containerID="abc4d1a07a30179925104ac09180347dc35a290683ae4dc1f1df1868fa600ba8"
Sep 17 11:21:20.695644 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.695526    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abc4d1a07a30179925104ac09180347dc35a290683ae4dc1f1df1868fa600ba8"} err="failed to get container status \"abc4d1a07a30179925104ac09180347dc35a290683ae4dc1f1df1868fa600ba8\": rpc error: code = NotFound desc = could not find container \"abc4d1a07a30179925104ac09180347dc35a290683ae4dc1f1df1868fa600ba8\": container with ID starting with abc4d1a07a30179925104ac09180347dc35a290683ae4dc1f1df1868fa600ba8 not found: ID does not exist"
Sep 17 11:21:20.695644 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.695569    2564 scope.go:117] "RemoveContainer" containerID="637f71d572671fefa4f5c950fd160f562c6f4e8e8d21516ce12bf501da245bfc"
Sep 17 11:21:20.696045 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:21:20.696018    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"637f71d572671fefa4f5c950fd160f562c6f4e8e8d21516ce12bf501da245bfc\": container with ID starting with 637f71d572671fefa4f5c950fd160f562c6f4e8e8d21516ce12bf501da245bfc not found: ID does not exist" containerID="637f71d572671fefa4f5c950fd160f562c6f4e8e8d21516ce12bf501da245bfc"
Sep 17 11:21:20.696137 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.696053    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"637f71d572671fefa4f5c950fd160f562c6f4e8e8d21516ce12bf501da245bfc"} err="failed to get container status \"637f71d572671fefa4f5c950fd160f562c6f4e8e8d21516ce12bf501da245bfc\": rpc error: code = NotFound desc = could not find container \"637f71d572671fefa4f5c950fd160f562c6f4e8e8d21516ce12bf501da245bfc\": container with ID starting with 637f71d572671fefa4f5c950fd160f562c6f4e8e8d21516ce12bf501da245bfc not found: ID does not exist"
Sep 17 11:21:20.698212 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.698191    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"]
Sep 17 11:21:20.703166 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:20.703144    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-9f97d7-openshift-default-5fdf996d6-zr94x"]
Sep 17 11:21:21.143361 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:21.143330    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f647c211-a46d-4ccb-8021-39a19888e795" path="/var/lib/kubelet/pods/f647c211-a46d-4ccb-8021-39a19888e795/volumes"
Sep 17 11:21:24.181804 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:24.181774    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-rl88q_995b3ba4-d76f-4b47-8194-940111359da7/payload-processing/0.log"
Sep 17 11:21:25.484074 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:25.484044    2564 scope.go:117] "RemoveContainer" containerID="90c0715e0fe0572b1ed516f8c7ee9ab08e6fad70c05aacb5505a5896c58bac65"
Sep 17 11:21:26.182180 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.182147    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"]
Sep 17 11:21:26.182419 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.182369    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt" podUID="a357c44c-eb18-444f-b16f-e489371ab850" containerName="payload-processing" containerID="cri-o://5d0f799b0e4154f19df2d92c170ebd8a77655ca8baec6b5f16a9c4b4b1ef052e" gracePeriod=30
Sep 17 11:21:26.192374 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.192344    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"]
Sep 17 11:21:26.192684 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.192640    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42" podUID="b8d2407c-c535-4c87-8bf7-6d7b0d91844a" containerName="payload-pre-processing" containerID="cri-o://8863f37276d8c37a2bb73d93734a1479e3e6d69a83df5f419cbbc785267f899b" gracePeriod=30
Sep 17 11:21:26.350443 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.350403    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq"]
Sep 17 11:21:26.353604 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.353581    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-7751cd-8g2gq"]
Sep 17 11:21:26.492575 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.492549    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"
Sep 17 11:21:26.495573 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.495554    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"
Sep 17 11:21:26.593577 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.593543    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a357c44c-eb18-444f-b16f-e489371ab850-plugins-config-volume\") pod \"a357c44c-eb18-444f-b16f-e489371ab850\" (UID: \"a357c44c-eb18-444f-b16f-e489371ab850\") "
Sep 17 11:21:26.593760 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.593617    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4nnr\" (UniqueName: \"kubernetes.io/projected/b8d2407c-c535-4c87-8bf7-6d7b0d91844a-kube-api-access-f4nnr\") pod \"b8d2407c-c535-4c87-8bf7-6d7b0d91844a\" (UID: \"b8d2407c-c535-4c87-8bf7-6d7b0d91844a\") "
Sep 17 11:21:26.593839 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.593820    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d2407c-c535-4c87-8bf7-6d7b0d91844a-plugins-config-volume\") pod \"b8d2407c-c535-4c87-8bf7-6d7b0d91844a\" (UID: \"b8d2407c-c535-4c87-8bf7-6d7b0d91844a\") "
Sep 17 11:21:26.593929 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.593854    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58rmk\" (UniqueName: \"kubernetes.io/projected/a357c44c-eb18-444f-b16f-e489371ab850-kube-api-access-58rmk\") pod \"a357c44c-eb18-444f-b16f-e489371ab850\" (UID: \"a357c44c-eb18-444f-b16f-e489371ab850\") "
Sep 17 11:21:26.593993 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.593849    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a357c44c-eb18-444f-b16f-e489371ab850-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a357c44c-eb18-444f-b16f-e489371ab850" (UID: "a357c44c-eb18-444f-b16f-e489371ab850"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:21:26.594201 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.594173    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a357c44c-eb18-444f-b16f-e489371ab850-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:26.594201 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.594189    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8d2407c-c535-4c87-8bf7-6d7b0d91844a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8d2407c-c535-4c87-8bf7-6d7b0d91844a" (UID: "b8d2407c-c535-4c87-8bf7-6d7b0d91844a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:21:26.595874 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.595850    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8d2407c-c535-4c87-8bf7-6d7b0d91844a-kube-api-access-f4nnr" (OuterVolumeSpecName: "kube-api-access-f4nnr") pod "b8d2407c-c535-4c87-8bf7-6d7b0d91844a" (UID: "b8d2407c-c535-4c87-8bf7-6d7b0d91844a"). InnerVolumeSpecName "kube-api-access-f4nnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:21:26.596039 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.596021    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a357c44c-eb18-444f-b16f-e489371ab850-kube-api-access-58rmk" (OuterVolumeSpecName: "kube-api-access-58rmk") pod "a357c44c-eb18-444f-b16f-e489371ab850" (UID: "a357c44c-eb18-444f-b16f-e489371ab850"). InnerVolumeSpecName "kube-api-access-58rmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:21:26.694954 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.694926    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d2407c-c535-4c87-8bf7-6d7b0d91844a-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:26.694954 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.694950    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58rmk\" (UniqueName: \"kubernetes.io/projected/a357c44c-eb18-444f-b16f-e489371ab850-kube-api-access-58rmk\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:26.694954 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.694961    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4nnr\" (UniqueName: \"kubernetes.io/projected/b8d2407c-c535-4c87-8bf7-6d7b0d91844a-kube-api-access-f4nnr\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:26.714341 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.714273    2564 generic.go:356] "Generic (PLEG): container finished" podID="a357c44c-eb18-444f-b16f-e489371ab850" containerID="5d0f799b0e4154f19df2d92c170ebd8a77655ca8baec6b5f16a9c4b4b1ef052e" exitCode=0
Sep 17 11:21:26.714341 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.714316    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"
Sep 17 11:21:26.714527 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.714365    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt" event={"ID":"a357c44c-eb18-444f-b16f-e489371ab850","Type":"ContainerDied","Data":"5d0f799b0e4154f19df2d92c170ebd8a77655ca8baec6b5f16a9c4b4b1ef052e"}
Sep 17 11:21:26.714527 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.714423    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt" event={"ID":"a357c44c-eb18-444f-b16f-e489371ab850","Type":"ContainerDied","Data":"ea3c28ad493b94f02563abf274ad45a53ff5395ca30d5e4b8dedcdcf43dadefe"}
Sep 17 11:21:26.714527 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.714454    2564 scope.go:117] "RemoveContainer" containerID="5d0f799b0e4154f19df2d92c170ebd8a77655ca8baec6b5f16a9c4b4b1ef052e"
Sep 17 11:21:26.715843 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.715824    2564 generic.go:356] "Generic (PLEG): container finished" podID="b8d2407c-c535-4c87-8bf7-6d7b0d91844a" containerID="8863f37276d8c37a2bb73d93734a1479e3e6d69a83df5f419cbbc785267f899b" exitCode=0
Sep 17 11:21:26.715968 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.715857    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42" event={"ID":"b8d2407c-c535-4c87-8bf7-6d7b0d91844a","Type":"ContainerDied","Data":"8863f37276d8c37a2bb73d93734a1479e3e6d69a83df5f419cbbc785267f899b"}
Sep 17 11:21:26.715968 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.715872    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"
Sep 17 11:21:26.716120 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.716103    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42" event={"ID":"b8d2407c-c535-4c87-8bf7-6d7b0d91844a","Type":"ContainerDied","Data":"7958d691fb184f52a5ebc827f42f43f0571b694088f6c417cbbd802988d7691a"}
Sep 17 11:21:26.725324 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.725302    2564 scope.go:117] "RemoveContainer" containerID="5d0f799b0e4154f19df2d92c170ebd8a77655ca8baec6b5f16a9c4b4b1ef052e"
Sep 17 11:21:26.725677 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:21:26.725569    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d0f799b0e4154f19df2d92c170ebd8a77655ca8baec6b5f16a9c4b4b1ef052e\": container with ID starting with 5d0f799b0e4154f19df2d92c170ebd8a77655ca8baec6b5f16a9c4b4b1ef052e not found: ID does not exist" containerID="5d0f799b0e4154f19df2d92c170ebd8a77655ca8baec6b5f16a9c4b4b1ef052e"
Sep 17 11:21:26.725677 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.725601    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d0f799b0e4154f19df2d92c170ebd8a77655ca8baec6b5f16a9c4b4b1ef052e"} err="failed to get container status \"5d0f799b0e4154f19df2d92c170ebd8a77655ca8baec6b5f16a9c4b4b1ef052e\": rpc error: code = NotFound desc = could not find container \"5d0f799b0e4154f19df2d92c170ebd8a77655ca8baec6b5f16a9c4b4b1ef052e\": container with ID starting with 5d0f799b0e4154f19df2d92c170ebd8a77655ca8baec6b5f16a9c4b4b1ef052e not found: ID does not exist"
Sep 17 11:21:26.725677 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.725619    2564 scope.go:117] "RemoveContainer" containerID="8863f37276d8c37a2bb73d93734a1479e3e6d69a83df5f419cbbc785267f899b"
Sep 17 11:21:26.733940 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.733915    2564 scope.go:117] "RemoveContainer" containerID="8863f37276d8c37a2bb73d93734a1479e3e6d69a83df5f419cbbc785267f899b"
Sep 17 11:21:26.734203 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:21:26.734186    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8863f37276d8c37a2bb73d93734a1479e3e6d69a83df5f419cbbc785267f899b\": container with ID starting with 8863f37276d8c37a2bb73d93734a1479e3e6d69a83df5f419cbbc785267f899b not found: ID does not exist" containerID="8863f37276d8c37a2bb73d93734a1479e3e6d69a83df5f419cbbc785267f899b"
Sep 17 11:21:26.734263 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.734210    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8863f37276d8c37a2bb73d93734a1479e3e6d69a83df5f419cbbc785267f899b"} err="failed to get container status \"8863f37276d8c37a2bb73d93734a1479e3e6d69a83df5f419cbbc785267f899b\": rpc error: code = NotFound desc = could not find container \"8863f37276d8c37a2bb73d93734a1479e3e6d69a83df5f419cbbc785267f899b\": container with ID starting with 8863f37276d8c37a2bb73d93734a1479e3e6d69a83df5f419cbbc785267f899b not found: ID does not exist"
Sep 17 11:21:26.739571 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.739551    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"]
Sep 17 11:21:26.747455 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.747435    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-7751cd-84c79cb4bf-444xt"]
Sep 17 11:21:26.757573 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.757553    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"]
Sep 17 11:21:26.763222 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:26.763198    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7751cd-66fc974cb6-4td42"]
Sep 17 11:21:27.142739 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:27.142707    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a357c44c-eb18-444f-b16f-e489371ab850" path="/var/lib/kubelet/pods/a357c44c-eb18-444f-b16f-e489371ab850/volumes"
Sep 17 11:21:27.143179 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:27.143163    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8d2407c-c535-4c87-8bf7-6d7b0d91844a" path="/var/lib/kubelet/pods/b8d2407c-c535-4c87-8bf7-6d7b0d91844a/volumes"
Sep 17 11:21:27.143475 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:27.143464    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8356b49-dcb8-4495-8755-daec0f256512" path="/var/lib/kubelet/pods/c8356b49-dcb8-4495-8755-daec0f256512/volumes"
Sep 17 11:21:33.655087 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:33.655047    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"]
Sep 17 11:21:33.655557 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:33.655393    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj" podUID="b9884649-aa25-414b-9912-d2a01cffe74d" containerName="istio-proxy" containerID="cri-o://c6ecd7d1057a4833fa205dc6ec0dbc8050c72ce29ea0f19641310ccccba88ad7" gracePeriod=30
Sep 17 11:21:34.008474 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.008436    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl"]
Sep 17 11:21:34.008838 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.008826    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f647c211-a46d-4ccb-8021-39a19888e795" containerName="istio-proxy"
Sep 17 11:21:34.008891 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.008840    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f647c211-a46d-4ccb-8021-39a19888e795" containerName="istio-proxy"
Sep 17 11:21:34.008891 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.008859    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a357c44c-eb18-444f-b16f-e489371ab850" containerName="payload-processing"
Sep 17 11:21:34.008891 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.008865    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a357c44c-eb18-444f-b16f-e489371ab850" containerName="payload-processing"
Sep 17 11:21:34.008891 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.008888    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8d2407c-c535-4c87-8bf7-6d7b0d91844a" containerName="payload-pre-processing"
Sep 17 11:21:34.008891 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.008909    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8d2407c-c535-4c87-8bf7-6d7b0d91844a" containerName="payload-pre-processing"
Sep 17 11:21:34.009074 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.008918    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8356b49-dcb8-4495-8755-daec0f256512" containerName="revoke-keys"
Sep 17 11:21:34.009074 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.008923    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8356b49-dcb8-4495-8755-daec0f256512" containerName="revoke-keys"
Sep 17 11:21:34.009074 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.008983    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8356b49-dcb8-4495-8755-daec0f256512" containerName="revoke-keys"
Sep 17 11:21:34.009074 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.008991    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f647c211-a46d-4ccb-8021-39a19888e795" containerName="istio-proxy"
Sep 17 11:21:34.009074 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.008998    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8d2407c-c535-4c87-8bf7-6d7b0d91844a" containerName="payload-pre-processing"
Sep 17 11:21:34.009074 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.009005    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a357c44c-eb18-444f-b16f-e489371ab850" containerName="payload-processing"
Sep 17 11:21:34.013683 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.013664    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl"
Sep 17 11:21:34.016060 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.016036    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:21:34.016174 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.016064    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:21:34.020944 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.020915    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl"]
Sep 17 11:21:34.167934 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.167887    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8c279a4-0c50-4bbd-a24d-1fe20a70bd90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl\" (UID: \"b8c279a4-0c50-4bbd-a24d-1fe20a70bd90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl"
Sep 17 11:21:34.268875 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.268778    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8c279a4-0c50-4bbd-a24d-1fe20a70bd90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl\" (UID: \"b8c279a4-0c50-4bbd-a24d-1fe20a70bd90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl"
Sep 17 11:21:34.269501 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.269480    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8c279a4-0c50-4bbd-a24d-1fe20a70bd90-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl\" (UID: \"b8c279a4-0c50-4bbd-a24d-1fe20a70bd90\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl"
Sep 17 11:21:34.325830 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.325803    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl"
Sep 17 11:21:34.618781 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.618751    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl"]
Sep 17 11:21:34.619672 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:21:34.619638    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c279a4_0c50_4bbd_a24d_1fe20a70bd90.slice/crio-1f11857f768e5daca504c719aa251bd90de514e080cd71b2eaaed28e1b29fa36 WatchSource:0}: Error finding container 1f11857f768e5daca504c719aa251bd90de514e080cd71b2eaaed28e1b29fa36: Status 404 returned error can't find the container with id 1f11857f768e5daca504c719aa251bd90de514e080cd71b2eaaed28e1b29fa36
Sep 17 11:21:34.695745 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.695718    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 17 11:21:34.696087 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.695781    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj" podUID="b9884649-aa25-414b-9912-d2a01cffe74d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 11:21:34.764953 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:34.764927    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl" event={"ID":"b8c279a4-0c50-4bbd-a24d-1fe20a70bd90","Type":"ContainerStarted","Data":"1f11857f768e5daca504c719aa251bd90de514e080cd71b2eaaed28e1b29fa36"}
Sep 17 11:21:35.772376 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:35.772336    2564 generic.go:356] "Generic (PLEG): container finished" podID="b8c279a4-0c50-4bbd-a24d-1fe20a70bd90" containerID="fce35d29d2ba8e7e64a0731c535993cf3c6fefe141f6f6e92e97ce8a4a939b0a" exitCode=0
Sep 17 11:21:35.772810 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:35.772423    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl" event={"ID":"b8c279a4-0c50-4bbd-a24d-1fe20a70bd90","Type":"ContainerDied","Data":"fce35d29d2ba8e7e64a0731c535993cf3c6fefe141f6f6e92e97ce8a4a939b0a"}
Sep 17 11:21:36.912673 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:36.912644    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl"
Sep 17 11:21:36.994732 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:36.994701    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8c279a4-0c50-4bbd-a24d-1fe20a70bd90-maas-api-service-ca\") pod \"b8c279a4-0c50-4bbd-a24d-1fe20a70bd90\" (UID: \"b8c279a4-0c50-4bbd-a24d-1fe20a70bd90\") "
Sep 17 11:21:36.995100 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:36.995076    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8c279a4-0c50-4bbd-a24d-1fe20a70bd90-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b8c279a4-0c50-4bbd-a24d-1fe20a70bd90" (UID: "b8c279a4-0c50-4bbd-a24d-1fe20a70bd90"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:21:37.095974 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:37.095881    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8c279a4-0c50-4bbd-a24d-1fe20a70bd90-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:37.786761 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:37.786729    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl"
Sep 17 11:21:37.786935 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:37.786774    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl" event={"ID":"b8c279a4-0c50-4bbd-a24d-1fe20a70bd90","Type":"ContainerDied","Data":"1f11857f768e5daca504c719aa251bd90de514e080cd71b2eaaed28e1b29fa36"}
Sep 17 11:21:37.786935 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:37.786820    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f11857f768e5daca504c719aa251bd90de514e080cd71b2eaaed28e1b29fa36"
Sep 17 11:21:38.136859 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:38.136765    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:21:38.795889 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:38.795858    2564 generic.go:356] "Generic (PLEG): container finished" podID="b9884649-aa25-414b-9912-d2a01cffe74d" containerID="c6ecd7d1057a4833fa205dc6ec0dbc8050c72ce29ea0f19641310ccccba88ad7" exitCode=0
Sep 17 11:21:38.796064 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:38.795931    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj" event={"ID":"b9884649-aa25-414b-9912-d2a01cffe74d","Type":"ContainerDied","Data":"c6ecd7d1057a4833fa205dc6ec0dbc8050c72ce29ea0f19641310ccccba88ad7"}
Sep 17 11:21:38.796064 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:38.796017    2564 scope.go:117] "RemoveContainer" containerID="ddd4eb6b43eadad0a5ddb2d1ba2fbb089558ad09b2ebc9cd4ada5d0a77fcbf30"
Sep 17 11:21:38.923757 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:38.923704    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:21:39.014241 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.014209    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9884649-aa25-414b-9912-d2a01cffe74d-istio-token\") pod \"b9884649-aa25-414b-9912-d2a01cffe74d\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") "
Sep 17 11:21:39.014374 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.014304    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-credential-socket\") pod \"b9884649-aa25-414b-9912-d2a01cffe74d\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") "
Sep 17 11:21:39.014374 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.014339    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9884649-aa25-414b-9912-d2a01cffe74d-istio-podinfo\") pod \"b9884649-aa25-414b-9912-d2a01cffe74d\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") "
Sep 17 11:21:39.014451 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.014388    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-istio-data\") pod \"b9884649-aa25-414b-9912-d2a01cffe74d\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") "
Sep 17 11:21:39.014451 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.014432    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-workload-socket\") pod \"b9884649-aa25-414b-9912-d2a01cffe74d\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") "
Sep 17 11:21:39.014547 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.014461    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9884649-aa25-414b-9912-d2a01cffe74d-istiod-ca-cert\") pod \"b9884649-aa25-414b-9912-d2a01cffe74d\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") "
Sep 17 11:21:39.014547 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.014497    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dsmm\" (UniqueName: \"kubernetes.io/projected/b9884649-aa25-414b-9912-d2a01cffe74d-kube-api-access-2dsmm\") pod \"b9884649-aa25-414b-9912-d2a01cffe74d\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") "
Sep 17 11:21:39.014547 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.014530    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-workload-certs\") pod \"b9884649-aa25-414b-9912-d2a01cffe74d\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") "
Sep 17 11:21:39.014702 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.014614    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b9884649-aa25-414b-9912-d2a01cffe74d" (UID: "b9884649-aa25-414b-9912-d2a01cffe74d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:21:39.014762 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.014708    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-istio-envoy\") pod \"b9884649-aa25-414b-9912-d2a01cffe74d\" (UID: \"b9884649-aa25-414b-9912-d2a01cffe74d\") "
Sep 17 11:21:39.014844 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.014703    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-istio-data" (OuterVolumeSpecName: "istio-data") pod "b9884649-aa25-414b-9912-d2a01cffe74d" (UID: "b9884649-aa25-414b-9912-d2a01cffe74d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:21:39.014940 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.014747    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b9884649-aa25-414b-9912-d2a01cffe74d" (UID: "b9884649-aa25-414b-9912-d2a01cffe74d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:21:39.014940 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.014776    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b9884649-aa25-414b-9912-d2a01cffe74d" (UID: "b9884649-aa25-414b-9912-d2a01cffe74d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:21:39.015180 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.015078    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9884649-aa25-414b-9912-d2a01cffe74d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b9884649-aa25-414b-9912-d2a01cffe74d" (UID: "b9884649-aa25-414b-9912-d2a01cffe74d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:21:39.015372 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.015307    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-credential-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:39.015372 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.015329    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-istio-data\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:39.015372 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.015344    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-workload-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:39.015372 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.015358    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9884649-aa25-414b-9912-d2a01cffe74d-istiod-ca-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:39.015372 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.015371    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-workload-certs\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:39.016583 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.016551    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9884649-aa25-414b-9912-d2a01cffe74d-istio-token" (OuterVolumeSpecName: "istio-token") pod "b9884649-aa25-414b-9912-d2a01cffe74d" (UID: "b9884649-aa25-414b-9912-d2a01cffe74d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:21:39.016680 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.016651    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b9884649-aa25-414b-9912-d2a01cffe74d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b9884649-aa25-414b-9912-d2a01cffe74d" (UID: "b9884649-aa25-414b-9912-d2a01cffe74d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 11:21:39.016781 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.016761    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b9884649-aa25-414b-9912-d2a01cffe74d" (UID: "b9884649-aa25-414b-9912-d2a01cffe74d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:21:39.016846 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.016826    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9884649-aa25-414b-9912-d2a01cffe74d-kube-api-access-2dsmm" (OuterVolumeSpecName: "kube-api-access-2dsmm") pod "b9884649-aa25-414b-9912-d2a01cffe74d" (UID: "b9884649-aa25-414b-9912-d2a01cffe74d"). InnerVolumeSpecName "kube-api-access-2dsmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:21:39.115878 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.115846    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9884649-aa25-414b-9912-d2a01cffe74d-istio-envoy\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:39.115878 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.115868    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9884649-aa25-414b-9912-d2a01cffe74d-istio-token\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:39.115878 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.115876    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9884649-aa25-414b-9912-d2a01cffe74d-istio-podinfo\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:39.115878 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.115884    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dsmm\" (UniqueName: \"kubernetes.io/projected/b9884649-aa25-414b-9912-d2a01cffe74d-kube-api-access-2dsmm\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:39.804244 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.804214    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"
Sep 17 11:21:39.804625 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.804242    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj" event={"ID":"b9884649-aa25-414b-9912-d2a01cffe74d","Type":"ContainerDied","Data":"f79fa9e28245b39a8318fa8399d250cc7ae30ca17aa91621f6af3a3a9d1f2d61"}
Sep 17 11:21:39.804625 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.804324    2564 scope.go:117] "RemoveContainer" containerID="c6ecd7d1057a4833fa205dc6ec0dbc8050c72ce29ea0f19641310ccccba88ad7"
Sep 17 11:21:39.822754 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.822732    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"]
Sep 17 11:21:39.826068 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:39.826045    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-7751cd-openshift-default-5b9dfd974-hktsj"]
Sep 17 11:21:41.143281 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:41.143247    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9884649-aa25-414b-9912-d2a01cffe74d" path="/var/lib/kubelet/pods/b9884649-aa25-414b-9912-d2a01cffe74d/volumes"
Sep 17 11:21:44.582835 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:44.582804    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"]
Sep 17 11:21:44.583326 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:44.583032    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw" podUID="5cfd6030-7451-40c5-984b-745fb276a505" containerName="payload-processing" containerID="cri-o://27d816c324a4f6796e2e8f62a76a2665a884c27ba77c9b939d69fbe786ce4df3" gracePeriod=30
Sep 17 11:21:44.840871 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:44.840842    2564 generic.go:356] "Generic (PLEG): container finished" podID="5cfd6030-7451-40c5-984b-745fb276a505" containerID="27d816c324a4f6796e2e8f62a76a2665a884c27ba77c9b939d69fbe786ce4df3" exitCode=0
Sep 17 11:21:44.841044 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:44.840937    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw" event={"ID":"5cfd6030-7451-40c5-984b-745fb276a505","Type":"ContainerDied","Data":"27d816c324a4f6796e2e8f62a76a2665a884c27ba77c9b939d69fbe786ce4df3"}
Sep 17 11:21:44.843488 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:44.843464    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl"]
Sep 17 11:21:44.846198 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:44.846176    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-947f83-92cwl"]
Sep 17 11:21:44.854034 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:44.854013    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"
Sep 17 11:21:44.876267 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:44.876244    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cfd6030-7451-40c5-984b-745fb276a505-plugins-config-volume\") pod \"5cfd6030-7451-40c5-984b-745fb276a505\" (UID: \"5cfd6030-7451-40c5-984b-745fb276a505\") "
Sep 17 11:21:44.876372 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:44.876324    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6nkh\" (UniqueName: \"kubernetes.io/projected/5cfd6030-7451-40c5-984b-745fb276a505-kube-api-access-s6nkh\") pod \"5cfd6030-7451-40c5-984b-745fb276a505\" (UID: \"5cfd6030-7451-40c5-984b-745fb276a505\") "
Sep 17 11:21:44.876646 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:44.876615    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cfd6030-7451-40c5-984b-745fb276a505-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5cfd6030-7451-40c5-984b-745fb276a505" (UID: "5cfd6030-7451-40c5-984b-745fb276a505"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:21:44.879338 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:44.879310    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cfd6030-7451-40c5-984b-745fb276a505-kube-api-access-s6nkh" (OuterVolumeSpecName: "kube-api-access-s6nkh") pod "5cfd6030-7451-40c5-984b-745fb276a505" (UID: "5cfd6030-7451-40c5-984b-745fb276a505"). InnerVolumeSpecName "kube-api-access-s6nkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:21:44.977367 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:44.977332    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6nkh\" (UniqueName: \"kubernetes.io/projected/5cfd6030-7451-40c5-984b-745fb276a505-kube-api-access-s6nkh\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:44.977367 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:44.977360    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cfd6030-7451-40c5-984b-745fb276a505-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:21:45.143815 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:45.143737    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8c279a4-0c50-4bbd-a24d-1fe20a70bd90" path="/var/lib/kubelet/pods/b8c279a4-0c50-4bbd-a24d-1fe20a70bd90/volumes"
Sep 17 11:21:45.848775 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:45.848744    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"
Sep 17 11:21:45.848775 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:45.848757    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw" event={"ID":"5cfd6030-7451-40c5-984b-745fb276a505","Type":"ContainerDied","Data":"beebe8f689a6a315c4196011913108d409ecb17a169ca1ef84b04eb815fdbb0a"}
Sep 17 11:21:45.849301 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:45.848846    2564 scope.go:117] "RemoveContainer" containerID="27d816c324a4f6796e2e8f62a76a2665a884c27ba77c9b939d69fbe786ce4df3"
Sep 17 11:21:45.867182 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:45.867158    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"]
Sep 17 11:21:45.878520 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:45.878496    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-947f83-5f86f66f99-kxljw"]
Sep 17 11:21:47.143130 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:21:47.143062    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cfd6030-7451-40c5-984b-745fb276a505" path="/var/lib/kubelet/pods/5cfd6030-7451-40c5-984b-745fb276a505/volumes"
Sep 17 11:22:20.759812 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:20.759784    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n_81613d36-c3b8-4a8a-889b-a3a663aa9e1e/payload-processing/0.log"
Sep 17 11:22:20.889512 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:20.889486    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-rl88q_995b3ba4-d76f-4b47-8194-940111359da7/payload-processing/0.log"
Sep 17 11:22:21.613579 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.613547    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"]
Sep 17 11:22:21.613951 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.613938    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9884649-aa25-414b-9912-d2a01cffe74d" containerName="istio-proxy"
Sep 17 11:22:21.614010 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.613953    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9884649-aa25-414b-9912-d2a01cffe74d" containerName="istio-proxy"
Sep 17 11:22:21.614010 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.613970    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8c279a4-0c50-4bbd-a24d-1fe20a70bd90" containerName="revoke-keys"
Sep 17 11:22:21.614010 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.613975    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8c279a4-0c50-4bbd-a24d-1fe20a70bd90" containerName="revoke-keys"
Sep 17 11:22:21.614010 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.613982    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9884649-aa25-414b-9912-d2a01cffe74d" containerName="istio-proxy"
Sep 17 11:22:21.614010 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.613986    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9884649-aa25-414b-9912-d2a01cffe74d" containerName="istio-proxy"
Sep 17 11:22:21.614010 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.613994    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f647c211-a46d-4ccb-8021-39a19888e795" containerName="istio-proxy"
Sep 17 11:22:21.614010 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.613999    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f647c211-a46d-4ccb-8021-39a19888e795" containerName="istio-proxy"
Sep 17 11:22:21.614010 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.614009    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cfd6030-7451-40c5-984b-745fb276a505" containerName="payload-processing"
Sep 17 11:22:21.614010 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.614015    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cfd6030-7451-40c5-984b-745fb276a505" containerName="payload-processing"
Sep 17 11:22:21.614318 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.614071    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9884649-aa25-414b-9912-d2a01cffe74d" containerName="istio-proxy"
Sep 17 11:22:21.614318 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.614077    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9884649-aa25-414b-9912-d2a01cffe74d" containerName="istio-proxy"
Sep 17 11:22:21.614318 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.614084    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8c279a4-0c50-4bbd-a24d-1fe20a70bd90" containerName="revoke-keys"
Sep 17 11:22:21.614318 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.614090    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f647c211-a46d-4ccb-8021-39a19888e795" containerName="istio-proxy"
Sep 17 11:22:21.614318 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.614096    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cfd6030-7451-40c5-984b-745fb276a505" containerName="payload-processing"
Sep 17 11:22:21.617059 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.617043    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.619460 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.619438    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-fd3842-openshift-default-dockercfg-6sclj\""
Sep 17 11:22:21.636051 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.636027    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"]
Sep 17 11:22:21.684414 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.684382    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e0246bd-0119-401a-9f6c-fc7602941826-istiod-ca-cert\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.684562 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.684422    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-credential-socket\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.684562 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.684457    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-istio-data\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.684562 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.684488    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-workload-certs\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.684562 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.684512    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-workload-socket\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.684562 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.684559    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-istio-envoy\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.684817 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.684581    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e0246bd-0119-401a-9f6c-fc7602941826-istio-podinfo\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.684817 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.684662    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e0246bd-0119-401a-9f6c-fc7602941826-istio-token\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.684817 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.684688    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbsk7\" (UniqueName: \"kubernetes.io/projected/1e0246bd-0119-401a-9f6c-fc7602941826-kube-api-access-lbsk7\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.785791 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.785758    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e0246bd-0119-401a-9f6c-fc7602941826-istiod-ca-cert\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.785791 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.785796    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-credential-socket\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.786303 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.785820    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-istio-data\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.786303 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.785845    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-workload-certs\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.786303 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.785870    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-workload-socket\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.786303 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.785893    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-istio-envoy\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.786303 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.785945    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e0246bd-0119-401a-9f6c-fc7602941826-istio-podinfo\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.786303 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.786033    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e0246bd-0119-401a-9f6c-fc7602941826-istio-token\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.786303 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.786061    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbsk7\" (UniqueName: \"kubernetes.io/projected/1e0246bd-0119-401a-9f6c-fc7602941826-kube-api-access-lbsk7\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.786303 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.786207    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-credential-socket\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.786303 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.786282    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-workload-certs\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.786747 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.786478    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-istio-data\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.786747 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.786577    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e0246bd-0119-401a-9f6c-fc7602941826-istiod-ca-cert\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.786747 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.786647    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-workload-socket\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.788581 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.788563    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-istio-envoy\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.788725 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.788708    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e0246bd-0119-401a-9f6c-fc7602941826-istio-podinfo\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.794124 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.794102    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e0246bd-0119-401a-9f6c-fc7602941826-istio-token\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.794288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.794270    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbsk7\" (UniqueName: \"kubernetes.io/projected/1e0246bd-0119-401a-9f6c-fc7602941826-kube-api-access-lbsk7\") pod \"e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") " pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:21.927427 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:21.927352    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:22.223420 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:22.223396    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"]
Sep 17 11:22:22.225304 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:22:22.225271    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e0246bd_0119_401a_9f6c_fc7602941826.slice/crio-671828f4cb56a5d689d082e46a28753397773d11ae153889be6f02177504f00d WatchSource:0}: Error finding container 671828f4cb56a5d689d082e46a28753397773d11ae153889be6f02177504f00d: Status 404 returned error can't find the container with id 671828f4cb56a5d689d082e46a28753397773d11ae153889be6f02177504f00d
Sep 17 11:22:22.227536 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:22.227500    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:22:22.227634 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:22.227564    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:22:22.227634 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:22.227594    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:22:23.067036 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:23.066994    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2" event={"ID":"1e0246bd-0119-401a-9f6c-fc7602941826","Type":"ContainerStarted","Data":"83ee3c85e8ecf0ed4165fb7eb95812467ef7a24740f078ee687b5a6ad6e4ebbc"}
Sep 17 11:22:23.067497 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:23.067058    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2" event={"ID":"1e0246bd-0119-401a-9f6c-fc7602941826","Type":"ContainerStarted","Data":"671828f4cb56a5d689d082e46a28753397773d11ae153889be6f02177504f00d"}
Sep 17 11:22:23.098068 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:23.098021    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2" podStartSLOduration=2.09800556 podStartE2EDuration="2.09800556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:22:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:22:23.094744773 +0000 UTC m=+1738.411107927" watchObservedRunningTime="2026-09-17 11:22:23.09800556 +0000 UTC m=+1738.414368715"
Sep 17 11:22:23.137313 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:23.137287    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:22:23.927979 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:23.927948    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:23.933867 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:23.933843    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:24.073269 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:24.073240    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:24.073982 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:24.073965    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:28.757825 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:28.757791    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"]
Sep 17 11:22:28.761634 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:28.761614    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"
Sep 17 11:22:28.764641 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:28.764620    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-fd3842-dockercfg-wjzxh\""
Sep 17 11:22:28.765051 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:28.765032    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-fd3842\""
Sep 17 11:22:28.781864 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:28.781838    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"]
Sep 17 11:22:28.854719 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:28.854680    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d8f043d-0451-40a6-93b6-2df4ae78a557-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc\" (UID: \"7d8f043d-0451-40a6-93b6-2df4ae78a557\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"
Sep 17 11:22:28.855025 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:28.854995    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mk4r\" (UniqueName: \"kubernetes.io/projected/7d8f043d-0451-40a6-93b6-2df4ae78a557-kube-api-access-6mk4r\") pod \"payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc\" (UID: \"7d8f043d-0451-40a6-93b6-2df4ae78a557\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"
Sep 17 11:22:28.955660 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:28.955631    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d8f043d-0451-40a6-93b6-2df4ae78a557-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc\" (UID: \"7d8f043d-0451-40a6-93b6-2df4ae78a557\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"
Sep 17 11:22:28.955795 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:28.955667    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mk4r\" (UniqueName: \"kubernetes.io/projected/7d8f043d-0451-40a6-93b6-2df4ae78a557-kube-api-access-6mk4r\") pod \"payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc\" (UID: \"7d8f043d-0451-40a6-93b6-2df4ae78a557\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"
Sep 17 11:22:28.956353 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:28.956331    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d8f043d-0451-40a6-93b6-2df4ae78a557-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc\" (UID: \"7d8f043d-0451-40a6-93b6-2df4ae78a557\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"
Sep 17 11:22:28.963568 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:28.963550    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mk4r\" (UniqueName: \"kubernetes.io/projected/7d8f043d-0451-40a6-93b6-2df4ae78a557-kube-api-access-6mk4r\") pod \"payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc\" (UID: \"7d8f043d-0451-40a6-93b6-2df4ae78a557\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"
Sep 17 11:22:29.071699 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:29.071644    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"
Sep 17 11:22:29.346409 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:29.346381    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"]
Sep 17 11:22:29.347400 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:22:29.347378    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d8f043d_0451_40a6_93b6_2df4ae78a557.slice/crio-a7ae1358f6e55906db3b2982c8ee8c16b26c33c372fe509202767adfe68cfd80 WatchSource:0}: Error finding container a7ae1358f6e55906db3b2982c8ee8c16b26c33c372fe509202767adfe68cfd80: Status 404 returned error can't find the container with id a7ae1358f6e55906db3b2982c8ee8c16b26c33c372fe509202767adfe68cfd80
Sep 17 11:22:30.114247 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:30.114213    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc" event={"ID":"7d8f043d-0451-40a6-93b6-2df4ae78a557","Type":"ContainerStarted","Data":"659d69866cece700078483571536bdb9888175c49dd16547576e0d0e857fd926"}
Sep 17 11:22:30.114630 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:30.114354    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"
Sep 17 11:22:30.114630 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:30.114375    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc" event={"ID":"7d8f043d-0451-40a6-93b6-2df4ae78a557","Type":"ContainerStarted","Data":"a7ae1358f6e55906db3b2982c8ee8c16b26c33c372fe509202767adfe68cfd80"}
Sep 17 11:22:30.134756 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:30.134698    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc" podStartSLOduration=2.13468215 podStartE2EDuration="2.13468215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:22:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:22:30.132957059 +0000 UTC m=+1745.449320215" watchObservedRunningTime="2026-09-17 11:22:30.13468215 +0000 UTC m=+1745.451045302"
Sep 17 11:22:41.129297 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:41.129268    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"
Sep 17 11:22:46.223303 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:46.223206    2564 generic.go:356] "Generic (PLEG): container finished" podID="1e0246bd-0119-401a-9f6c-fc7602941826" containerID="83ee3c85e8ecf0ed4165fb7eb95812467ef7a24740f078ee687b5a6ad6e4ebbc" exitCode=0
Sep 17 11:22:46.223303 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:46.223279    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2" event={"ID":"1e0246bd-0119-401a-9f6c-fc7602941826","Type":"ContainerDied","Data":"83ee3c85e8ecf0ed4165fb7eb95812467ef7a24740f078ee687b5a6ad6e4ebbc"}
Sep 17 11:22:46.223740 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:46.223693    2564 scope.go:117] "RemoveContainer" containerID="83ee3c85e8ecf0ed4165fb7eb95812467ef7a24740f078ee687b5a6ad6e4ebbc"
Sep 17 11:22:46.224785 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:46.224752    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:22:46.224843 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:46.224824    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:22:46.224891 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:46.224862    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 17 11:22:47.230691 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:47.230658    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2" event={"ID":"1e0246bd-0119-401a-9f6c-fc7602941826","Type":"ContainerStarted","Data":"60061c99aac5cca3b17571448b8a39389f8e22c191319a72d046ab465b9106f2"}
Sep 17 11:22:47.928573 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:47.928533    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:47.928776 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:47.928755    2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Sep 17 11:22:47.928836 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:47.928816    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2" podUID="1e0246bd-0119-401a-9f6c-fc7602941826" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Sep 17 11:22:48.928520 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:48.928484    2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Sep 17 11:22:48.928977 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:48.928543    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2" podUID="1e0246bd-0119-401a-9f6c-fc7602941826" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Sep 17 11:22:49.928323 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:49.928286    2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Sep 17 11:22:49.928513 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:49.928339    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2" podUID="1e0246bd-0119-401a-9f6c-fc7602941826" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Sep 17 11:22:50.928203 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:50.928168    2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Sep 17 11:22:50.928618 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:50.928224    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2" podUID="1e0246bd-0119-401a-9f6c-fc7602941826" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Sep 17 11:22:51.928183 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:51.928148    2564 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Sep 17 11:22:51.928375 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:51.928209    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2" podUID="1e0246bd-0119-401a-9f6c-fc7602941826" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Sep 17 11:22:52.931993 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:52.931955    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:52.932379 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:52.932303    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:52.933086 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:52.933069    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:22:55.805932 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:55.805867    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6"]
Sep 17 11:22:55.808729 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:55.808708    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6"
Sep 17 11:22:55.811858 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:55.811841    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:22:55.812400 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:55.812387    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:22:55.818385 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:55.818363    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6"]
Sep 17 11:22:55.908813 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:55.908776    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2619bdc4-b4c4-4400-bfb9-45b1638386c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6\" (UID: \"2619bdc4-b4c4-4400-bfb9-45b1638386c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6"
Sep 17 11:22:56.009876 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:56.009839    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2619bdc4-b4c4-4400-bfb9-45b1638386c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6\" (UID: \"2619bdc4-b4c4-4400-bfb9-45b1638386c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6"
Sep 17 11:22:56.010467 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:56.010449    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2619bdc4-b4c4-4400-bfb9-45b1638386c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6\" (UID: \"2619bdc4-b4c4-4400-bfb9-45b1638386c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6"
Sep 17 11:22:56.120196 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:56.120117    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6"
Sep 17 11:22:56.396862 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:56.396838    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6"]
Sep 17 11:22:56.398999 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:22:56.398976    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2619bdc4_b4c4_4400_bfb9_45b1638386c1.slice/crio-62cb8584d7f813caff9c1d27b2214e23a3ad9e6578bdbd8443d8330d8ea83e60 WatchSource:0}: Error finding container 62cb8584d7f813caff9c1d27b2214e23a3ad9e6578bdbd8443d8330d8ea83e60: Status 404 returned error can't find the container with id 62cb8584d7f813caff9c1d27b2214e23a3ad9e6578bdbd8443d8330d8ea83e60
Sep 17 11:22:57.286662 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:57.286630    2564 generic.go:356] "Generic (PLEG): container finished" podID="2619bdc4-b4c4-4400-bfb9-45b1638386c1" containerID="d2e43bd0251ce0981cdbc72f682e6ad4a44955bf26ccc866609859589529da58" exitCode=0
Sep 17 11:22:57.287038 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:57.286693    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6" event={"ID":"2619bdc4-b4c4-4400-bfb9-45b1638386c1","Type":"ContainerDied","Data":"d2e43bd0251ce0981cdbc72f682e6ad4a44955bf26ccc866609859589529da58"}
Sep 17 11:22:57.287038 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:57.286745    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6" event={"ID":"2619bdc4-b4c4-4400-bfb9-45b1638386c1","Type":"ContainerStarted","Data":"62cb8584d7f813caff9c1d27b2214e23a3ad9e6578bdbd8443d8330d8ea83e60"}
Sep 17 11:22:58.432084 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:58.432063    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6"
Sep 17 11:22:58.533247 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:58.533215    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2619bdc4-b4c4-4400-bfb9-45b1638386c1-maas-api-service-ca\") pod \"2619bdc4-b4c4-4400-bfb9-45b1638386c1\" (UID: \"2619bdc4-b4c4-4400-bfb9-45b1638386c1\") "
Sep 17 11:22:58.533613 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:58.533591    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2619bdc4-b4c4-4400-bfb9-45b1638386c1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2619bdc4-b4c4-4400-bfb9-45b1638386c1" (UID: "2619bdc4-b4c4-4400-bfb9-45b1638386c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:22:58.634123 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:58.634046    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2619bdc4-b4c4-4400-bfb9-45b1638386c1-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:22:59.300459 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:59.300423    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6" event={"ID":"2619bdc4-b4c4-4400-bfb9-45b1638386c1","Type":"ContainerDied","Data":"62cb8584d7f813caff9c1d27b2214e23a3ad9e6578bdbd8443d8330d8ea83e60"}
Sep 17 11:22:59.300641 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:59.300472    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62cb8584d7f813caff9c1d27b2214e23a3ad9e6578bdbd8443d8330d8ea83e60"
Sep 17 11:22:59.300641 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:22:59.300468    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6"
Sep 17 11:23:06.905729 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:06.905018    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"]
Sep 17 11:23:06.905729 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:06.905299    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc" podUID="7d8f043d-0451-40a6-93b6-2df4ae78a557" containerName="payload-processing" containerID="cri-o://659d69866cece700078483571536bdb9888175c49dd16547576e0d0e857fd926" gracePeriod=30
Sep 17 11:23:07.094225 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.094145    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6"]
Sep 17 11:23:07.097645 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.097615    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fd3842-g6nz6"]
Sep 17 11:23:07.137165 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.137138    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:23:07.144039 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.144019    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2619bdc4-b4c4-4400-bfb9-45b1638386c1" path="/var/lib/kubelet/pods/2619bdc4-b4c4-4400-bfb9-45b1638386c1/volumes"
Sep 17 11:23:07.164727 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.164677    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"
Sep 17 11:23:07.319104 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.319069    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mk4r\" (UniqueName: \"kubernetes.io/projected/7d8f043d-0451-40a6-93b6-2df4ae78a557-kube-api-access-6mk4r\") pod \"7d8f043d-0451-40a6-93b6-2df4ae78a557\" (UID: \"7d8f043d-0451-40a6-93b6-2df4ae78a557\") "
Sep 17 11:23:07.319306 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.319141    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d8f043d-0451-40a6-93b6-2df4ae78a557-plugins-config-volume\") pod \"7d8f043d-0451-40a6-93b6-2df4ae78a557\" (UID: \"7d8f043d-0451-40a6-93b6-2df4ae78a557\") "
Sep 17 11:23:07.319444 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.319420    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d8f043d-0451-40a6-93b6-2df4ae78a557-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7d8f043d-0451-40a6-93b6-2df4ae78a557" (UID: "7d8f043d-0451-40a6-93b6-2df4ae78a557"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:23:07.321331 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.321306    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d8f043d-0451-40a6-93b6-2df4ae78a557-kube-api-access-6mk4r" (OuterVolumeSpecName: "kube-api-access-6mk4r") pod "7d8f043d-0451-40a6-93b6-2df4ae78a557" (UID: "7d8f043d-0451-40a6-93b6-2df4ae78a557"). InnerVolumeSpecName "kube-api-access-6mk4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:23:07.350992 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.350967    2564 generic.go:356] "Generic (PLEG): container finished" podID="7d8f043d-0451-40a6-93b6-2df4ae78a557" containerID="659d69866cece700078483571536bdb9888175c49dd16547576e0d0e857fd926" exitCode=0
Sep 17 11:23:07.351087 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.351018    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"
Sep 17 11:23:07.351087 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.351055    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc" event={"ID":"7d8f043d-0451-40a6-93b6-2df4ae78a557","Type":"ContainerDied","Data":"659d69866cece700078483571536bdb9888175c49dd16547576e0d0e857fd926"}
Sep 17 11:23:07.351161 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.351106    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc" event={"ID":"7d8f043d-0451-40a6-93b6-2df4ae78a557","Type":"ContainerDied","Data":"a7ae1358f6e55906db3b2982c8ee8c16b26c33c372fe509202767adfe68cfd80"}
Sep 17 11:23:07.351161 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.351127    2564 scope.go:117] "RemoveContainer" containerID="659d69866cece700078483571536bdb9888175c49dd16547576e0d0e857fd926"
Sep 17 11:23:07.360639 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.360623    2564 scope.go:117] "RemoveContainer" containerID="659d69866cece700078483571536bdb9888175c49dd16547576e0d0e857fd926"
Sep 17 11:23:07.360956 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:23:07.360889    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"659d69866cece700078483571536bdb9888175c49dd16547576e0d0e857fd926\": container with ID starting with 659d69866cece700078483571536bdb9888175c49dd16547576e0d0e857fd926 not found: ID does not exist" containerID="659d69866cece700078483571536bdb9888175c49dd16547576e0d0e857fd926"
Sep 17 11:23:07.361015 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.360971    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"659d69866cece700078483571536bdb9888175c49dd16547576e0d0e857fd926"} err="failed to get container status \"659d69866cece700078483571536bdb9888175c49dd16547576e0d0e857fd926\": rpc error: code = NotFound desc = could not find container \"659d69866cece700078483571536bdb9888175c49dd16547576e0d0e857fd926\": container with ID starting with 659d69866cece700078483571536bdb9888175c49dd16547576e0d0e857fd926 not found: ID does not exist"
Sep 17 11:23:07.381560 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.381540    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"]
Sep 17 11:23:07.391100 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.391081    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fd3842-77f66db76f-jrpmc"]
Sep 17 11:23:07.420763 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.420716    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d8f043d-0451-40a6-93b6-2df4ae78a557-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:23:07.420763 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:07.420733    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mk4r\" (UniqueName: \"kubernetes.io/projected/7d8f043d-0451-40a6-93b6-2df4ae78a557-kube-api-access-6mk4r\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:23:09.143889 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:09.143852    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d8f043d-0451-40a6-93b6-2df4ae78a557" path="/var/lib/kubelet/pods/7d8f043d-0451-40a6-93b6-2df4ae78a557/volumes"
Sep 17 11:23:14.341218 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:14.341185    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"]
Sep 17 11:23:14.341670 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:14.341397    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2" podUID="1e0246bd-0119-401a-9f6c-fc7602941826" containerName="istio-proxy" containerID="cri-o://60061c99aac5cca3b17571448b8a39389f8e22c191319a72d046ab465b9106f2" gracePeriod=30
Sep 17 11:23:18.036510 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:18.036430    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v"]
Sep 17 11:23:18.036921 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:18.036880    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2619bdc4-b4c4-4400-bfb9-45b1638386c1" containerName="revoke-keys"
Sep 17 11:23:18.036921 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:18.036915    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2619bdc4-b4c4-4400-bfb9-45b1638386c1" containerName="revoke-keys"
Sep 17 11:23:18.037021 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:18.036930    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d8f043d-0451-40a6-93b6-2df4ae78a557" containerName="payload-processing"
Sep 17 11:23:18.037021 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:18.036938    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d8f043d-0451-40a6-93b6-2df4ae78a557" containerName="payload-processing"
Sep 17 11:23:18.037097 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:18.037030    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d8f043d-0451-40a6-93b6-2df4ae78a557" containerName="payload-processing"
Sep 17 11:23:18.037097 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:18.037041    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2619bdc4-b4c4-4400-bfb9-45b1638386c1" containerName="revoke-keys"
Sep 17 11:23:18.046864 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:18.046834    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v"
Sep 17 11:23:18.049427 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:18.049390    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v"]
Sep 17 11:23:18.049819 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:18.049800    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:23:18.050030 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:18.049814    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:23:18.123459 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:18.123420    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f583c79-b888-47b1-82c9-987a91eec274-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v\" (UID: \"5f583c79-b888-47b1-82c9-987a91eec274\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v"
Sep 17 11:23:18.224444 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:18.224396    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f583c79-b888-47b1-82c9-987a91eec274-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v\" (UID: \"5f583c79-b888-47b1-82c9-987a91eec274\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v"
Sep 17 11:23:18.225041 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:18.225022    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f583c79-b888-47b1-82c9-987a91eec274-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v\" (UID: \"5f583c79-b888-47b1-82c9-987a91eec274\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v"
Sep 17 11:23:18.358982 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:18.358866    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v"
Sep 17 11:23:18.641370 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:18.641300    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v"]
Sep 17 11:23:18.644525 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:23:18.644501    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f583c79_b888_47b1_82c9_987a91eec274.slice/crio-29c3876da282ff0c34a24b2bb7b91ba25680c332ce0be460c7bd167c7c50bd19 WatchSource:0}: Error finding container 29c3876da282ff0c34a24b2bb7b91ba25680c332ce0be460c7bd167c7c50bd19: Status 404 returned error can't find the container with id 29c3876da282ff0c34a24b2bb7b91ba25680c332ce0be460c7bd167c7c50bd19
Sep 17 11:23:19.430675 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.430637    2564 generic.go:356] "Generic (PLEG): container finished" podID="5f583c79-b888-47b1-82c9-987a91eec274" containerID="2f40ea655079dbca81a464cb45b922022443cf4df83592f66d0d5c2f91b2f07e" exitCode=0
Sep 17 11:23:19.431163 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.430727    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v" event={"ID":"5f583c79-b888-47b1-82c9-987a91eec274","Type":"ContainerDied","Data":"2f40ea655079dbca81a464cb45b922022443cf4df83592f66d0d5c2f91b2f07e"}
Sep 17 11:23:19.431163 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.430780    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v" event={"ID":"5f583c79-b888-47b1-82c9-987a91eec274","Type":"ContainerStarted","Data":"29c3876da282ff0c34a24b2bb7b91ba25680c332ce0be460c7bd167c7c50bd19"}
Sep 17 11:23:19.601278 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.601248    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:23:19.637473 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.637450    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-credential-socket\") pod \"1e0246bd-0119-401a-9f6c-fc7602941826\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") "
Sep 17 11:23:19.637677 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.637484    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-istio-data\") pod \"1e0246bd-0119-401a-9f6c-fc7602941826\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") "
Sep 17 11:23:19.637677 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.637510    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbsk7\" (UniqueName: \"kubernetes.io/projected/1e0246bd-0119-401a-9f6c-fc7602941826-kube-api-access-lbsk7\") pod \"1e0246bd-0119-401a-9f6c-fc7602941826\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") "
Sep 17 11:23:19.637677 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.637539    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e0246bd-0119-401a-9f6c-fc7602941826-istio-podinfo\") pod \"1e0246bd-0119-401a-9f6c-fc7602941826\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") "
Sep 17 11:23:19.637677 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.637574    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e0246bd-0119-401a-9f6c-fc7602941826-istiod-ca-cert\") pod \"1e0246bd-0119-401a-9f6c-fc7602941826\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") "
Sep 17 11:23:19.637677 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.637607    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-workload-socket\") pod \"1e0246bd-0119-401a-9f6c-fc7602941826\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") "
Sep 17 11:23:19.637677 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.637632    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-workload-certs\") pod \"1e0246bd-0119-401a-9f6c-fc7602941826\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") "
Sep 17 11:23:19.637677 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.637658    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-istio-envoy\") pod \"1e0246bd-0119-401a-9f6c-fc7602941826\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") "
Sep 17 11:23:19.638112 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.637683    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e0246bd-0119-401a-9f6c-fc7602941826-istio-token\") pod \"1e0246bd-0119-401a-9f6c-fc7602941826\" (UID: \"1e0246bd-0119-401a-9f6c-fc7602941826\") "
Sep 17 11:23:19.638112 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.637743    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1e0246bd-0119-401a-9f6c-fc7602941826" (UID: "1e0246bd-0119-401a-9f6c-fc7602941826"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:23:19.638112 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.637959    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1e0246bd-0119-401a-9f6c-fc7602941826" (UID: "1e0246bd-0119-401a-9f6c-fc7602941826"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:23:19.638112 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.638066    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-workload-certs\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:23:19.638112 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.638086    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-credential-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:23:19.638385 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.638226    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1e0246bd-0119-401a-9f6c-fc7602941826" (UID: "1e0246bd-0119-401a-9f6c-fc7602941826"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:23:19.638385 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.638270    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e0246bd-0119-401a-9f6c-fc7602941826-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1e0246bd-0119-401a-9f6c-fc7602941826" (UID: "1e0246bd-0119-401a-9f6c-fc7602941826"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:23:19.638385 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.638347    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-istio-data" (OuterVolumeSpecName: "istio-data") pod "1e0246bd-0119-401a-9f6c-fc7602941826" (UID: "1e0246bd-0119-401a-9f6c-fc7602941826"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:23:19.640320 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.640297    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e0246bd-0119-401a-9f6c-fc7602941826-istio-token" (OuterVolumeSpecName: "istio-token") pod "1e0246bd-0119-401a-9f6c-fc7602941826" (UID: "1e0246bd-0119-401a-9f6c-fc7602941826"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:23:19.640453 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.640343    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1e0246bd-0119-401a-9f6c-fc7602941826-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1e0246bd-0119-401a-9f6c-fc7602941826" (UID: "1e0246bd-0119-401a-9f6c-fc7602941826"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 11:23:19.640453 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.640399    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1e0246bd-0119-401a-9f6c-fc7602941826" (UID: "1e0246bd-0119-401a-9f6c-fc7602941826"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:23:19.640453 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.640413    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e0246bd-0119-401a-9f6c-fc7602941826-kube-api-access-lbsk7" (OuterVolumeSpecName: "kube-api-access-lbsk7") pod "1e0246bd-0119-401a-9f6c-fc7602941826" (UID: "1e0246bd-0119-401a-9f6c-fc7602941826"). InnerVolumeSpecName "kube-api-access-lbsk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:23:19.739071 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.739039    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e0246bd-0119-401a-9f6c-fc7602941826-istiod-ca-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:23:19.739071 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.739065    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-workload-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:23:19.739071 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.739074    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-istio-envoy\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:23:19.739285 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.739083    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e0246bd-0119-401a-9f6c-fc7602941826-istio-token\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:23:19.739285 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.739092    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e0246bd-0119-401a-9f6c-fc7602941826-istio-data\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:23:19.739285 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.739100    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbsk7\" (UniqueName: \"kubernetes.io/projected/1e0246bd-0119-401a-9f6c-fc7602941826-kube-api-access-lbsk7\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:23:19.739285 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:19.739110    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e0246bd-0119-401a-9f6c-fc7602941826-istio-podinfo\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:23:20.437919 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:20.437859    2564 generic.go:356] "Generic (PLEG): container finished" podID="1e0246bd-0119-401a-9f6c-fc7602941826" containerID="60061c99aac5cca3b17571448b8a39389f8e22c191319a72d046ab465b9106f2" exitCode=0
Sep 17 11:23:20.438395 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:20.437938    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"
Sep 17 11:23:20.438395 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:20.437960    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2" event={"ID":"1e0246bd-0119-401a-9f6c-fc7602941826","Type":"ContainerDied","Data":"60061c99aac5cca3b17571448b8a39389f8e22c191319a72d046ab465b9106f2"}
Sep 17 11:23:20.438395 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:20.438031    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2" event={"ID":"1e0246bd-0119-401a-9f6c-fc7602941826","Type":"ContainerDied","Data":"671828f4cb56a5d689d082e46a28753397773d11ae153889be6f02177504f00d"}
Sep 17 11:23:20.438395 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:20.438069    2564 scope.go:117] "RemoveContainer" containerID="60061c99aac5cca3b17571448b8a39389f8e22c191319a72d046ab465b9106f2"
Sep 17 11:23:20.448716 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:20.448697    2564 scope.go:117] "RemoveContainer" containerID="83ee3c85e8ecf0ed4165fb7eb95812467ef7a24740f078ee687b5a6ad6e4ebbc"
Sep 17 11:23:20.458704 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:20.458680    2564 scope.go:117] "RemoveContainer" containerID="60061c99aac5cca3b17571448b8a39389f8e22c191319a72d046ab465b9106f2"
Sep 17 11:23:20.459074 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:23:20.459049    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60061c99aac5cca3b17571448b8a39389f8e22c191319a72d046ab465b9106f2\": container with ID starting with 60061c99aac5cca3b17571448b8a39389f8e22c191319a72d046ab465b9106f2 not found: ID does not exist" containerID="60061c99aac5cca3b17571448b8a39389f8e22c191319a72d046ab465b9106f2"
Sep 17 11:23:20.459160 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:20.459086    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60061c99aac5cca3b17571448b8a39389f8e22c191319a72d046ab465b9106f2"} err="failed to get container status \"60061c99aac5cca3b17571448b8a39389f8e22c191319a72d046ab465b9106f2\": rpc error: code = NotFound desc = could not find container \"60061c99aac5cca3b17571448b8a39389f8e22c191319a72d046ab465b9106f2\": container with ID starting with 60061c99aac5cca3b17571448b8a39389f8e22c191319a72d046ab465b9106f2 not found: ID does not exist"
Sep 17 11:23:20.459160 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:20.459111    2564 scope.go:117] "RemoveContainer" containerID="83ee3c85e8ecf0ed4165fb7eb95812467ef7a24740f078ee687b5a6ad6e4ebbc"
Sep 17 11:23:20.459443 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:23:20.459415    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83ee3c85e8ecf0ed4165fb7eb95812467ef7a24740f078ee687b5a6ad6e4ebbc\": container with ID starting with 83ee3c85e8ecf0ed4165fb7eb95812467ef7a24740f078ee687b5a6ad6e4ebbc not found: ID does not exist" containerID="83ee3c85e8ecf0ed4165fb7eb95812467ef7a24740f078ee687b5a6ad6e4ebbc"
Sep 17 11:23:20.459520 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:20.459451    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83ee3c85e8ecf0ed4165fb7eb95812467ef7a24740f078ee687b5a6ad6e4ebbc"} err="failed to get container status \"83ee3c85e8ecf0ed4165fb7eb95812467ef7a24740f078ee687b5a6ad6e4ebbc\": rpc error: code = NotFound desc = could not find container \"83ee3c85e8ecf0ed4165fb7eb95812467ef7a24740f078ee687b5a6ad6e4ebbc\": container with ID starting with 83ee3c85e8ecf0ed4165fb7eb95812467ef7a24740f078ee687b5a6ad6e4ebbc not found: ID does not exist"
Sep 17 11:23:20.463449 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:20.463380    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"]
Sep 17 11:23:20.466733 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:20.466715    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-fd3842-openshift-default-74c45b894-cttt2"]
Sep 17 11:23:20.582618 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:20.582594    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v"
Sep 17 11:23:20.648792 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:20.648760    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f583c79-b888-47b1-82c9-987a91eec274-maas-api-service-ca\") pod \"5f583c79-b888-47b1-82c9-987a91eec274\" (UID: \"5f583c79-b888-47b1-82c9-987a91eec274\") "
Sep 17 11:23:20.649141 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:20.649116    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f583c79-b888-47b1-82c9-987a91eec274-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f583c79-b888-47b1-82c9-987a91eec274" (UID: "5f583c79-b888-47b1-82c9-987a91eec274"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:23:20.749716 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:20.749693    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f583c79-b888-47b1-82c9-987a91eec274-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:23:21.143172 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:21.143093    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e0246bd-0119-401a-9f6c-fc7602941826" path="/var/lib/kubelet/pods/1e0246bd-0119-401a-9f6c-fc7602941826/volumes"
Sep 17 11:23:21.446427 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:21.446345    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v" event={"ID":"5f583c79-b888-47b1-82c9-987a91eec274","Type":"ContainerDied","Data":"29c3876da282ff0c34a24b2bb7b91ba25680c332ce0be460c7bd167c7c50bd19"}
Sep 17 11:23:21.446427 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:21.446385    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v"
Sep 17 11:23:21.446828 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:21.446390    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29c3876da282ff0c34a24b2bb7b91ba25680c332ce0be460c7bd167c7c50bd19"
Sep 17 11:23:25.549044 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:25.549012    2564 scope.go:117] "RemoveContainer" containerID="4c7e9f6bbeeb8ebc6e6009982b2b9c7a151fcd66e05369bf821ea9262664c1b8"
Sep 17 11:23:33.759707 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:33.759675    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"]
Sep 17 11:23:33.760113 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:33.759873    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n" podUID="81613d36-c3b8-4a8a-889b-a3a663aa9e1e" containerName="payload-processing" containerID="cri-o://572b0cd15ede155cb0bf1bf97e07f6878f13e29d2d5a885db4cd532b22f43514" gracePeriod=30
Sep 17 11:23:34.015580 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.015556    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"
Sep 17 11:23:34.070913 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.070870    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sb5z\" (UniqueName: \"kubernetes.io/projected/81613d36-c3b8-4a8a-889b-a3a663aa9e1e-kube-api-access-5sb5z\") pod \"81613d36-c3b8-4a8a-889b-a3a663aa9e1e\" (UID: \"81613d36-c3b8-4a8a-889b-a3a663aa9e1e\") "
Sep 17 11:23:34.071068 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.070961    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81613d36-c3b8-4a8a-889b-a3a663aa9e1e-plugins-config-volume\") pod \"81613d36-c3b8-4a8a-889b-a3a663aa9e1e\" (UID: \"81613d36-c3b8-4a8a-889b-a3a663aa9e1e\") "
Sep 17 11:23:34.071291 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.071268    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81613d36-c3b8-4a8a-889b-a3a663aa9e1e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "81613d36-c3b8-4a8a-889b-a3a663aa9e1e" (UID: "81613d36-c3b8-4a8a-889b-a3a663aa9e1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:23:34.073291 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.073266    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81613d36-c3b8-4a8a-889b-a3a663aa9e1e-kube-api-access-5sb5z" (OuterVolumeSpecName: "kube-api-access-5sb5z") pod "81613d36-c3b8-4a8a-889b-a3a663aa9e1e" (UID: "81613d36-c3b8-4a8a-889b-a3a663aa9e1e"). InnerVolumeSpecName "kube-api-access-5sb5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:23:34.172300 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.172199    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sb5z\" (UniqueName: \"kubernetes.io/projected/81613d36-c3b8-4a8a-889b-a3a663aa9e1e-kube-api-access-5sb5z\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:23:34.172300 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.172234    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81613d36-c3b8-4a8a-889b-a3a663aa9e1e-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:23:34.174782 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.174757    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v"]
Sep 17 11:23:34.179870 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.179849    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-31fca0-2bl2v"]
Sep 17 11:23:34.528340 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.528308    2564 generic.go:356] "Generic (PLEG): container finished" podID="81613d36-c3b8-4a8a-889b-a3a663aa9e1e" containerID="572b0cd15ede155cb0bf1bf97e07f6878f13e29d2d5a885db4cd532b22f43514" exitCode=0
Sep 17 11:23:34.528518 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.528347    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"
Sep 17 11:23:34.528518 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.528373    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n" event={"ID":"81613d36-c3b8-4a8a-889b-a3a663aa9e1e","Type":"ContainerDied","Data":"572b0cd15ede155cb0bf1bf97e07f6878f13e29d2d5a885db4cd532b22f43514"}
Sep 17 11:23:34.528518 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.528415    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n" event={"ID":"81613d36-c3b8-4a8a-889b-a3a663aa9e1e","Type":"ContainerDied","Data":"b2b460a8e4bab2cb01712994c87ce06152a098a2f3b9c7cb9dd53c67642ea526"}
Sep 17 11:23:34.528518 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.528436    2564 scope.go:117] "RemoveContainer" containerID="572b0cd15ede155cb0bf1bf97e07f6878f13e29d2d5a885db4cd532b22f43514"
Sep 17 11:23:34.538442 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.538422    2564 scope.go:117] "RemoveContainer" containerID="572b0cd15ede155cb0bf1bf97e07f6878f13e29d2d5a885db4cd532b22f43514"
Sep 17 11:23:34.538715 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:23:34.538694    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"572b0cd15ede155cb0bf1bf97e07f6878f13e29d2d5a885db4cd532b22f43514\": container with ID starting with 572b0cd15ede155cb0bf1bf97e07f6878f13e29d2d5a885db4cd532b22f43514 not found: ID does not exist" containerID="572b0cd15ede155cb0bf1bf97e07f6878f13e29d2d5a885db4cd532b22f43514"
Sep 17 11:23:34.538764 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.538726    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"572b0cd15ede155cb0bf1bf97e07f6878f13e29d2d5a885db4cd532b22f43514"} err="failed to get container status \"572b0cd15ede155cb0bf1bf97e07f6878f13e29d2d5a885db4cd532b22f43514\": rpc error: code = NotFound desc = could not find container \"572b0cd15ede155cb0bf1bf97e07f6878f13e29d2d5a885db4cd532b22f43514\": container with ID starting with 572b0cd15ede155cb0bf1bf97e07f6878f13e29d2d5a885db4cd532b22f43514 not found: ID does not exist"
Sep 17 11:23:34.578858 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.578828    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"]
Sep 17 11:23:34.679337 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:34.679308    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-31fca0-85b785c4fc-r4s9n"]
Sep 17 11:23:35.143415 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:35.143384    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f583c79-b888-47b1-82c9-987a91eec274" path="/var/lib/kubelet/pods/5f583c79-b888-47b1-82c9-987a91eec274/volumes"
Sep 17 11:23:35.143773 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:35.143721    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81613d36-c3b8-4a8a-889b-a3a663aa9e1e" path="/var/lib/kubelet/pods/81613d36-c3b8-4a8a-889b-a3a663aa9e1e/volumes"
Sep 17 11:23:49.137299 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:23:49.137267    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:24:25.033034 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.033000    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"]
Sep 17 11:24:25.033541 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.033182    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng" podUID="8a9c5855-f6e1-48c7-a4ea-cf703069a72f" containerName="istio-proxy" containerID="cri-o://da34249788cc6e548a193b61be5812d60d9f08f620d28b5262f7f64f9406092d" gracePeriod=30
Sep 17 11:24:25.375439 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.375345    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9"]
Sep 17 11:24:25.375819 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.375800    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f583c79-b888-47b1-82c9-987a91eec274" containerName="revoke-keys"
Sep 17 11:24:25.375937 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.375821    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f583c79-b888-47b1-82c9-987a91eec274" containerName="revoke-keys"
Sep 17 11:24:25.375937 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.375833    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81613d36-c3b8-4a8a-889b-a3a663aa9e1e" containerName="payload-processing"
Sep 17 11:24:25.375937 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.375841    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="81613d36-c3b8-4a8a-889b-a3a663aa9e1e" containerName="payload-processing"
Sep 17 11:24:25.375937 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.375865    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e0246bd-0119-401a-9f6c-fc7602941826" containerName="istio-proxy"
Sep 17 11:24:25.375937 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.375873    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e0246bd-0119-401a-9f6c-fc7602941826" containerName="istio-proxy"
Sep 17 11:24:25.375937 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.375887    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e0246bd-0119-401a-9f6c-fc7602941826" containerName="istio-proxy"
Sep 17 11:24:25.375937 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.375914    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e0246bd-0119-401a-9f6c-fc7602941826" containerName="istio-proxy"
Sep 17 11:24:25.376285 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.376034    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f583c79-b888-47b1-82c9-987a91eec274" containerName="revoke-keys"
Sep 17 11:24:25.376285 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.376050    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e0246bd-0119-401a-9f6c-fc7602941826" containerName="istio-proxy"
Sep 17 11:24:25.376285 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.376059    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="81613d36-c3b8-4a8a-889b-a3a663aa9e1e" containerName="payload-processing"
Sep 17 11:24:25.376285 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.376069    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e0246bd-0119-401a-9f6c-fc7602941826" containerName="istio-proxy"
Sep 17 11:24:25.379301 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.379279    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9"
Sep 17 11:24:25.381366 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.381348    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:24:25.381366 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.381360    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:24:25.388401 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.388368    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9"]
Sep 17 11:24:25.439509 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.439478    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cdba72a-38fa-480b-af78-db133b15fffe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9\" (UID: \"9cdba72a-38fa-480b-af78-db133b15fffe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9"
Sep 17 11:24:25.540014 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.539980    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cdba72a-38fa-480b-af78-db133b15fffe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9\" (UID: \"9cdba72a-38fa-480b-af78-db133b15fffe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9"
Sep 17 11:24:25.540576 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.540557    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cdba72a-38fa-480b-af78-db133b15fffe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9\" (UID: \"9cdba72a-38fa-480b-af78-db133b15fffe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9"
Sep 17 11:24:25.690443 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.690362    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9"
Sep 17 11:24:25.967202 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:25.967174    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9"]
Sep 17 11:24:25.969600 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:24:25.969544    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cdba72a_38fa_480b_af78_db133b15fffe.slice/crio-bd7531dbc94e98487abff295a51af7e95e380b362f2eb6b5527e15f08b75bb5c WatchSource:0}: Error finding container bd7531dbc94e98487abff295a51af7e95e380b362f2eb6b5527e15f08b75bb5c: Status 404 returned error can't find the container with id bd7531dbc94e98487abff295a51af7e95e380b362f2eb6b5527e15f08b75bb5c
Sep 17 11:24:26.828489 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:26.828451    2564 generic.go:356] "Generic (PLEG): container finished" podID="9cdba72a-38fa-480b-af78-db133b15fffe" containerID="cb0351ebeba2bc7f61c010f291e54271129c5df8646d2d62fe50ea4d6ed1154e" exitCode=0
Sep 17 11:24:26.828860 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:26.828518    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9" event={"ID":"9cdba72a-38fa-480b-af78-db133b15fffe","Type":"ContainerDied","Data":"cb0351ebeba2bc7f61c010f291e54271129c5df8646d2d62fe50ea4d6ed1154e"}
Sep 17 11:24:26.828860 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:26.828612    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9" event={"ID":"9cdba72a-38fa-480b-af78-db133b15fffe","Type":"ContainerStarted","Data":"bd7531dbc94e98487abff295a51af7e95e380b362f2eb6b5527e15f08b75bb5c"}
Sep 17 11:24:27.967730 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:27.967706    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9"
Sep 17 11:24:28.065877 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:28.065848    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cdba72a-38fa-480b-af78-db133b15fffe-maas-api-service-ca\") pod \"9cdba72a-38fa-480b-af78-db133b15fffe\" (UID: \"9cdba72a-38fa-480b-af78-db133b15fffe\") "
Sep 17 11:24:28.066202 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:28.066178    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cdba72a-38fa-480b-af78-db133b15fffe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9cdba72a-38fa-480b-af78-db133b15fffe" (UID: "9cdba72a-38fa-480b-af78-db133b15fffe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:24:28.167396 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:28.167307    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cdba72a-38fa-480b-af78-db133b15fffe-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:24:28.841060 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:28.841031    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9"
Sep 17 11:24:28.841060 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:28.841046    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9" event={"ID":"9cdba72a-38fa-480b-af78-db133b15fffe","Type":"ContainerDied","Data":"bd7531dbc94e98487abff295a51af7e95e380b362f2eb6b5527e15f08b75bb5c"}
Sep 17 11:24:28.841279 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:28.841091    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd7531dbc94e98487abff295a51af7e95e380b362f2eb6b5527e15f08b75bb5c"
Sep 17 11:24:30.290719 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.290697    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:24:30.385039 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.384974    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-token\") pod \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") "
Sep 17 11:24:30.385176 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.385051    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-data\") pod \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") "
Sep 17 11:24:30.385176 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.385144    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-workload-socket\") pod \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") "
Sep 17 11:24:30.385250 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.385190    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istiod-ca-cert\") pod \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") "
Sep 17 11:24:30.385250 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.385214    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-credential-socket\") pod \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") "
Sep 17 11:24:30.385250 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.385245    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-envoy\") pod \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") "
Sep 17 11:24:30.385377 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.385299    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rgqd\" (UniqueName: \"kubernetes.io/projected/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-kube-api-access-6rgqd\") pod \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") "
Sep 17 11:24:30.385377 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.385300    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-data" (OuterVolumeSpecName: "istio-data") pod "8a9c5855-f6e1-48c7-a4ea-cf703069a72f" (UID: "8a9c5855-f6e1-48c7-a4ea-cf703069a72f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:24:30.385377 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.385353    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-podinfo\") pod \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") "
Sep 17 11:24:30.385534 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.385363    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8a9c5855-f6e1-48c7-a4ea-cf703069a72f" (UID: "8a9c5855-f6e1-48c7-a4ea-cf703069a72f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:24:30.385534 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.385378    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-workload-certs\") pod \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\" (UID: \"8a9c5855-f6e1-48c7-a4ea-cf703069a72f\") "
Sep 17 11:24:30.385643 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.385597    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8a9c5855-f6e1-48c7-a4ea-cf703069a72f" (UID: "8a9c5855-f6e1-48c7-a4ea-cf703069a72f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:24:30.385643 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.385610    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8a9c5855-f6e1-48c7-a4ea-cf703069a72f" (UID: "8a9c5855-f6e1-48c7-a4ea-cf703069a72f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:24:30.385788 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.385768    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-workload-certs\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:24:30.385850 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.385791    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-data\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:24:30.385850 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.385800    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-workload-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:24:30.385850 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.385808    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istiod-ca-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:24:30.386207 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.386033    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8a9c5855-f6e1-48c7-a4ea-cf703069a72f" (UID: "8a9c5855-f6e1-48c7-a4ea-cf703069a72f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:24:30.387378 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.387354    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-token" (OuterVolumeSpecName: "istio-token") pod "8a9c5855-f6e1-48c7-a4ea-cf703069a72f" (UID: "8a9c5855-f6e1-48c7-a4ea-cf703069a72f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:24:30.387553 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.387535    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-kube-api-access-6rgqd" (OuterVolumeSpecName: "kube-api-access-6rgqd") pod "8a9c5855-f6e1-48c7-a4ea-cf703069a72f" (UID: "8a9c5855-f6e1-48c7-a4ea-cf703069a72f"). InnerVolumeSpecName "kube-api-access-6rgqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:24:30.387628 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.387576    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8a9c5855-f6e1-48c7-a4ea-cf703069a72f" (UID: "8a9c5855-f6e1-48c7-a4ea-cf703069a72f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:24:30.387857 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.387827    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8a9c5855-f6e1-48c7-a4ea-cf703069a72f" (UID: "8a9c5855-f6e1-48c7-a4ea-cf703069a72f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 11:24:30.487181 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.487146    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-credential-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:24:30.487181 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.487173    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-envoy\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:24:30.487181 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.487188    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rgqd\" (UniqueName: \"kubernetes.io/projected/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-kube-api-access-6rgqd\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:24:30.487395 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.487200    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-podinfo\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:24:30.487395 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.487212    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a9c5855-f6e1-48c7-a4ea-cf703069a72f-istio-token\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:24:30.853709 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.853670    2564 generic.go:356] "Generic (PLEG): container finished" podID="8a9c5855-f6e1-48c7-a4ea-cf703069a72f" containerID="da34249788cc6e548a193b61be5812d60d9f08f620d28b5262f7f64f9406092d" exitCode=0
Sep 17 11:24:30.853878 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.853719    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"
Sep 17 11:24:30.853878 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.853749    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng" event={"ID":"8a9c5855-f6e1-48c7-a4ea-cf703069a72f","Type":"ContainerDied","Data":"da34249788cc6e548a193b61be5812d60d9f08f620d28b5262f7f64f9406092d"}
Sep 17 11:24:30.853878 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.853805    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng" event={"ID":"8a9c5855-f6e1-48c7-a4ea-cf703069a72f","Type":"ContainerDied","Data":"16c29375c89db6746d125564ecb8dc7fd6862df1910ad0824e80fd957bbe8285"}
Sep 17 11:24:30.853878 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.853830    2564 scope.go:117] "RemoveContainer" containerID="da34249788cc6e548a193b61be5812d60d9f08f620d28b5262f7f64f9406092d"
Sep 17 11:24:30.863697 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.863683    2564 scope.go:117] "RemoveContainer" containerID="face60727f7637d3dfc38df7697ba04d289ecfe8749ad3516f031629b0ed89bf"
Sep 17 11:24:30.872079 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.872064    2564 scope.go:117] "RemoveContainer" containerID="da34249788cc6e548a193b61be5812d60d9f08f620d28b5262f7f64f9406092d"
Sep 17 11:24:30.872322 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:24:30.872305    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da34249788cc6e548a193b61be5812d60d9f08f620d28b5262f7f64f9406092d\": container with ID starting with da34249788cc6e548a193b61be5812d60d9f08f620d28b5262f7f64f9406092d not found: ID does not exist" containerID="da34249788cc6e548a193b61be5812d60d9f08f620d28b5262f7f64f9406092d"
Sep 17 11:24:30.872376 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.872330    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da34249788cc6e548a193b61be5812d60d9f08f620d28b5262f7f64f9406092d"} err="failed to get container status \"da34249788cc6e548a193b61be5812d60d9f08f620d28b5262f7f64f9406092d\": rpc error: code = NotFound desc = could not find container \"da34249788cc6e548a193b61be5812d60d9f08f620d28b5262f7f64f9406092d\": container with ID starting with da34249788cc6e548a193b61be5812d60d9f08f620d28b5262f7f64f9406092d not found: ID does not exist"
Sep 17 11:24:30.872376 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.872346    2564 scope.go:117] "RemoveContainer" containerID="face60727f7637d3dfc38df7697ba04d289ecfe8749ad3516f031629b0ed89bf"
Sep 17 11:24:30.872601 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:24:30.872585    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"face60727f7637d3dfc38df7697ba04d289ecfe8749ad3516f031629b0ed89bf\": container with ID starting with face60727f7637d3dfc38df7697ba04d289ecfe8749ad3516f031629b0ed89bf not found: ID does not exist" containerID="face60727f7637d3dfc38df7697ba04d289ecfe8749ad3516f031629b0ed89bf"
Sep 17 11:24:30.872643 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.872605    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"face60727f7637d3dfc38df7697ba04d289ecfe8749ad3516f031629b0ed89bf"} err="failed to get container status \"face60727f7637d3dfc38df7697ba04d289ecfe8749ad3516f031629b0ed89bf\": rpc error: code = NotFound desc = could not find container \"face60727f7637d3dfc38df7697ba04d289ecfe8749ad3516f031629b0ed89bf\": container with ID starting with face60727f7637d3dfc38df7697ba04d289ecfe8749ad3516f031629b0ed89bf not found: ID does not exist"
Sep 17 11:24:30.876307 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.876288    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"]
Sep 17 11:24:30.881365 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:30.881342    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-31fca0-openshift-default-b7bb4bc8-rp6ng"]
Sep 17 11:24:31.136806 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:31.136717    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:24:31.142435 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:31.142403    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a9c5855-f6e1-48c7-a4ea-cf703069a72f" path="/var/lib/kubelet/pods/8a9c5855-f6e1-48c7-a4ea-cf703069a72f/volumes"
Sep 17 11:24:36.251827 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:36.251795    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9"]
Sep 17 11:24:36.255806 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:36.255781    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-23a45b-2fxg9"]
Sep 17 11:24:37.143344 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:37.143311    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cdba72a-38fa-480b-af78-db133b15fffe" path="/var/lib/kubelet/pods/9cdba72a-38fa-480b-af78-db133b15fffe/volumes"
Sep 17 11:24:43.206764 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.206727    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d"]
Sep 17 11:24:43.207159 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.207111    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cdba72a-38fa-480b-af78-db133b15fffe" containerName="revoke-keys"
Sep 17 11:24:43.207159 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.207124    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cdba72a-38fa-480b-af78-db133b15fffe" containerName="revoke-keys"
Sep 17 11:24:43.207159 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.207134    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a9c5855-f6e1-48c7-a4ea-cf703069a72f" containerName="istio-proxy"
Sep 17 11:24:43.207159 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.207139    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a9c5855-f6e1-48c7-a4ea-cf703069a72f" containerName="istio-proxy"
Sep 17 11:24:43.207297 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.207214    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a9c5855-f6e1-48c7-a4ea-cf703069a72f" containerName="istio-proxy"
Sep 17 11:24:43.207297 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.207224    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cdba72a-38fa-480b-af78-db133b15fffe" containerName="revoke-keys"
Sep 17 11:24:43.211604 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.211587    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d"
Sep 17 11:24:43.214226 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.214206    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:24:43.214312 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.214281    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:24:43.223137 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.223114    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d"]
Sep 17 11:24:43.304559 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.304528    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26e39ea8-fdd7-4666-9855-4b80041a5684-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d\" (UID: \"26e39ea8-fdd7-4666-9855-4b80041a5684\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d"
Sep 17 11:24:43.405553 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.405519    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26e39ea8-fdd7-4666-9855-4b80041a5684-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d\" (UID: \"26e39ea8-fdd7-4666-9855-4b80041a5684\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d"
Sep 17 11:24:43.406151 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.406132    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26e39ea8-fdd7-4666-9855-4b80041a5684-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d\" (UID: \"26e39ea8-fdd7-4666-9855-4b80041a5684\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d"
Sep 17 11:24:43.522541 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.522513    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d"
Sep 17 11:24:43.807127 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.807098    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d"]
Sep 17 11:24:43.809967 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:24:43.809938    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26e39ea8_fdd7_4666_9855_4b80041a5684.slice/crio-bf7f60aad9f15373976340ba41d7fdaa9d1bacae000551c561c607e2af87a032 WatchSource:0}: Error finding container bf7f60aad9f15373976340ba41d7fdaa9d1bacae000551c561c607e2af87a032: Status 404 returned error can't find the container with id bf7f60aad9f15373976340ba41d7fdaa9d1bacae000551c561c607e2af87a032
Sep 17 11:24:43.933135 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:43.933098    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d" event={"ID":"26e39ea8-fdd7-4666-9855-4b80041a5684","Type":"ContainerStarted","Data":"bf7f60aad9f15373976340ba41d7fdaa9d1bacae000551c561c607e2af87a032"}
Sep 17 11:24:44.940991 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:44.940875    2564 generic.go:356] "Generic (PLEG): container finished" podID="26e39ea8-fdd7-4666-9855-4b80041a5684" containerID="77d465e79a719bba766a57c40715562464450123a9b120a41fe68f2e2f992742" exitCode=0
Sep 17 11:24:44.940991 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:44.940959    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d" event={"ID":"26e39ea8-fdd7-4666-9855-4b80041a5684","Type":"ContainerDied","Data":"77d465e79a719bba766a57c40715562464450123a9b120a41fe68f2e2f992742"}
Sep 17 11:24:46.074716 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:46.074693    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d"
Sep 17 11:24:46.125936 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:46.125890    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26e39ea8-fdd7-4666-9855-4b80041a5684-maas-api-service-ca\") pod \"26e39ea8-fdd7-4666-9855-4b80041a5684\" (UID: \"26e39ea8-fdd7-4666-9855-4b80041a5684\") "
Sep 17 11:24:46.126273 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:46.126244    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26e39ea8-fdd7-4666-9855-4b80041a5684-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "26e39ea8-fdd7-4666-9855-4b80041a5684" (UID: "26e39ea8-fdd7-4666-9855-4b80041a5684"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:24:46.227336 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:46.227265    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26e39ea8-fdd7-4666-9855-4b80041a5684-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:24:46.960106 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:46.960079    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d"
Sep 17 11:24:46.960260 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:46.960119    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d" event={"ID":"26e39ea8-fdd7-4666-9855-4b80041a5684","Type":"ContainerDied","Data":"bf7f60aad9f15373976340ba41d7fdaa9d1bacae000551c561c607e2af87a032"}
Sep 17 11:24:46.960260 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:46.960186    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf7f60aad9f15373976340ba41d7fdaa9d1bacae000551c561c607e2af87a032"
Sep 17 11:24:53.842634 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:53.842596    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-026f86-29827395-z5hdk"]
Sep 17 11:24:53.847389 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:53.847364    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-026f86-29827395-z5hdk"]
Sep 17 11:24:53.889597 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:53.889570    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"]
Sep 17 11:24:53.889785 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:53.889762    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858" podUID="b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da" containerName="payload-processing" containerID="cri-o://6b0840db9449c1c2a2794dced168cbd7e9bf921f16d94c23cd9c89e2f0b3fa23" gracePeriod=30
Sep 17 11:24:54.010465 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:54.010433    2564 generic.go:356] "Generic (PLEG): container finished" podID="b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da" containerID="6b0840db9449c1c2a2794dced168cbd7e9bf921f16d94c23cd9c89e2f0b3fa23" exitCode=0
Sep 17 11:24:54.010576 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:54.010507    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858" event={"ID":"b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da","Type":"ContainerDied","Data":"6b0840db9449c1c2a2794dced168cbd7e9bf921f16d94c23cd9c89e2f0b3fa23"}
Sep 17 11:24:54.111685 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:54.111610    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d"]
Sep 17 11:24:54.117201 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:54.117177    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-026f86-gcl6d"]
Sep 17 11:24:54.147498 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:54.147476    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"
Sep 17 11:24:54.201053 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:54.201020    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da-plugins-config-volume\") pod \"b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da\" (UID: \"b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da\") "
Sep 17 11:24:54.201233 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:54.201150    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdkjr\" (UniqueName: \"kubernetes.io/projected/b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da-kube-api-access-vdkjr\") pod \"b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da\" (UID: \"b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da\") "
Sep 17 11:24:54.201416 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:54.201394    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da" (UID: "b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:24:54.203506 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:54.203479    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da-kube-api-access-vdkjr" (OuterVolumeSpecName: "kube-api-access-vdkjr") pod "b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da" (UID: "b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da"). InnerVolumeSpecName "kube-api-access-vdkjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:24:54.302661 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:54.302626    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdkjr\" (UniqueName: \"kubernetes.io/projected/b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da-kube-api-access-vdkjr\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:24:54.302661 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:54.302653    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:24:55.018032 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:55.017997    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"
Sep 17 11:24:55.018425 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:55.018035    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858" event={"ID":"b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da","Type":"ContainerDied","Data":"ac554cdc473a820c11feb2d6c8279bf6ad546d34afd4bc163ea90041e89b6867"}
Sep 17 11:24:55.018425 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:55.018097    2564 scope.go:117] "RemoveContainer" containerID="6b0840db9449c1c2a2794dced168cbd7e9bf921f16d94c23cd9c89e2f0b3fa23"
Sep 17 11:24:55.041208 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:55.041183    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"]
Sep 17 11:24:55.045238 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:55.045213    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-026f86-86cd76fb97-h9858"]
Sep 17 11:24:55.144191 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:55.144160    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26e39ea8-fdd7-4666-9855-4b80041a5684" path="/var/lib/kubelet/pods/26e39ea8-fdd7-4666-9855-4b80041a5684/volumes"
Sep 17 11:24:55.144452 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:55.144439    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="985defe8-ac78-4a5b-9ce9-a9a26b4e5665" path="/var/lib/kubelet/pods/985defe8-ac78-4a5b-9ce9-a9a26b4e5665/volumes"
Sep 17 11:24:55.144714 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:24:55.144703    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da" path="/var/lib/kubelet/pods/b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da/volumes"
Sep 17 11:25:00.923964 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:00.923932    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"]
Sep 17 11:25:00.924509 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:00.924136    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml" podUID="0be46ac4-4bc1-4975-9223-791bd5028769" containerName="istio-proxy" containerID="cri-o://b674cbc7d599f381cd8736fd91bda99f4cf58b1b1c71c76b82285f5da8384f41" gracePeriod=30
Sep 17 11:25:01.291744 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.291707    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9"]
Sep 17 11:25:01.292089 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.292074    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da" containerName="payload-processing"
Sep 17 11:25:01.292148 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.292090    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da" containerName="payload-processing"
Sep 17 11:25:01.292148 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.292104    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26e39ea8-fdd7-4666-9855-4b80041a5684" containerName="revoke-keys"
Sep 17 11:25:01.292148 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.292110    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="26e39ea8-fdd7-4666-9855-4b80041a5684" containerName="revoke-keys"
Sep 17 11:25:01.292148 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.292126    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a9c5855-f6e1-48c7-a4ea-cf703069a72f" containerName="istio-proxy"
Sep 17 11:25:01.292148 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.292132    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a9c5855-f6e1-48c7-a4ea-cf703069a72f" containerName="istio-proxy"
Sep 17 11:25:01.292357 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.292192    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a9c5855-f6e1-48c7-a4ea-cf703069a72f" containerName="istio-proxy"
Sep 17 11:25:01.292357 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.292199    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="26e39ea8-fdd7-4666-9855-4b80041a5684" containerName="revoke-keys"
Sep 17 11:25:01.292357 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.292205    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b298ab07-bbbd-4d62-b6ee-8f91ddfdb5da" containerName="payload-processing"
Sep 17 11:25:01.295054 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.295037    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9"
Sep 17 11:25:01.297501 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.297476    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 11:25:01.297630 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.297523    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:25:01.304885 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.304853    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9"]
Sep 17 11:25:01.364350 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.364321    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30911f8c-0066-4817-8e8c-d466a073f456-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9\" (UID: \"30911f8c-0066-4817-8e8c-d466a073f456\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9"
Sep 17 11:25:01.465018 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.464987    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30911f8c-0066-4817-8e8c-d466a073f456-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9\" (UID: \"30911f8c-0066-4817-8e8c-d466a073f456\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9"
Sep 17 11:25:01.465571 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.465551    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30911f8c-0066-4817-8e8c-d466a073f456-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9\" (UID: \"30911f8c-0066-4817-8e8c-d466a073f456\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9"
Sep 17 11:25:01.608115 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.608040    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9"
Sep 17 11:25:01.885026 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:01.884995    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9"]
Sep 17 11:25:01.886672 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:25:01.886643    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30911f8c_0066_4817_8e8c_d466a073f456.slice/crio-64ac26978a036dcbd326f509cdf00197ded50c7a451335b4d39fd05ddf98a4e2 WatchSource:0}: Error finding container 64ac26978a036dcbd326f509cdf00197ded50c7a451335b4d39fd05ddf98a4e2: Status 404 returned error can't find the container with id 64ac26978a036dcbd326f509cdf00197ded50c7a451335b4d39fd05ddf98a4e2
Sep 17 11:25:02.069109 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:02.069071    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9" event={"ID":"30911f8c-0066-4817-8e8c-d466a073f456","Type":"ContainerStarted","Data":"64ac26978a036dcbd326f509cdf00197ded50c7a451335b4d39fd05ddf98a4e2"}
Sep 17 11:25:03.076256 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:03.076224    2564 generic.go:356] "Generic (PLEG): container finished" podID="30911f8c-0066-4817-8e8c-d466a073f456" containerID="81685373ed487d106a24d07ebdbb6c796cfe07b8ae836394eaed1cca1e2f51e7" exitCode=0
Sep 17 11:25:03.076644 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:03.076299    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9" event={"ID":"30911f8c-0066-4817-8e8c-d466a073f456","Type":"ContainerDied","Data":"81685373ed487d106a24d07ebdbb6c796cfe07b8ae836394eaed1cca1e2f51e7"}
Sep 17 11:25:03.591150 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:03.591114    2564 patch_prober.go:28] interesting pod/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 17 11:25:03.591313 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:03.591175    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml" podUID="0be46ac4-4bc1-4975-9223-791bd5028769" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 11:25:04.213075 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:04.213053    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9"
Sep 17 11:25:04.290160 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:04.290133    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30911f8c-0066-4817-8e8c-d466a073f456-maas-api-service-ca\") pod \"30911f8c-0066-4817-8e8c-d466a073f456\" (UID: \"30911f8c-0066-4817-8e8c-d466a073f456\") "
Sep 17 11:25:04.290472 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:04.290450    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30911f8c-0066-4817-8e8c-d466a073f456-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "30911f8c-0066-4817-8e8c-d466a073f456" (UID: "30911f8c-0066-4817-8e8c-d466a073f456"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:25:04.391276 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:04.391245    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30911f8c-0066-4817-8e8c-d466a073f456-maas-api-service-ca\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:25:05.088867 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:05.088840    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9"
Sep 17 11:25:05.088867 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:05.088846    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9" event={"ID":"30911f8c-0066-4817-8e8c-d466a073f456","Type":"ContainerDied","Data":"64ac26978a036dcbd326f509cdf00197ded50c7a451335b4d39fd05ddf98a4e2"}
Sep 17 11:25:05.089095 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:05.088915    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64ac26978a036dcbd326f509cdf00197ded50c7a451335b4d39fd05ddf98a4e2"
Sep 17 11:25:06.099015 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.098981    2564 generic.go:356] "Generic (PLEG): container finished" podID="0be46ac4-4bc1-4975-9223-791bd5028769" containerID="b674cbc7d599f381cd8736fd91bda99f4cf58b1b1c71c76b82285f5da8384f41" exitCode=0
Sep 17 11:25:06.099400 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.099050    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml" event={"ID":"0be46ac4-4bc1-4975-9223-791bd5028769","Type":"ContainerDied","Data":"b674cbc7d599f381cd8736fd91bda99f4cf58b1b1c71c76b82285f5da8384f41"}
Sep 17 11:25:06.099400 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.099132    2564 scope.go:117] "RemoveContainer" containerID="b7147623e0d98f1fbbac68c1e62fede13922de9a2d787cf113c736f3113f3e37"
Sep 17 11:25:06.164625 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.164603    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:25:06.306757 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.306733    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-workload-socket\") pod \"0be46ac4-4bc1-4975-9223-791bd5028769\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") "
Sep 17 11:25:06.306950 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.306795    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-istio-envoy\") pod \"0be46ac4-4bc1-4975-9223-791bd5028769\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") "
Sep 17 11:25:06.306950 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.306817    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be46ac4-4bc1-4975-9223-791bd5028769-istiod-ca-cert\") pod \"0be46ac4-4bc1-4975-9223-791bd5028769\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") "
Sep 17 11:25:06.306950 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.306852    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be46ac4-4bc1-4975-9223-791bd5028769-istio-token\") pod \"0be46ac4-4bc1-4975-9223-791bd5028769\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") "
Sep 17 11:25:06.306950 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.306874    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-credential-socket\") pod \"0be46ac4-4bc1-4975-9223-791bd5028769\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") "
Sep 17 11:25:06.306950 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.306918    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-istio-data\") pod \"0be46ac4-4bc1-4975-9223-791bd5028769\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") "
Sep 17 11:25:06.307257 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.306981    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be46ac4-4bc1-4975-9223-791bd5028769-istio-podinfo\") pod \"0be46ac4-4bc1-4975-9223-791bd5028769\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") "
Sep 17 11:25:06.307257 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.307018    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0be46ac4-4bc1-4975-9223-791bd5028769" (UID: "0be46ac4-4bc1-4975-9223-791bd5028769"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:25:06.307257 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.307024    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb66x\" (UniqueName: \"kubernetes.io/projected/0be46ac4-4bc1-4975-9223-791bd5028769-kube-api-access-kb66x\") pod \"0be46ac4-4bc1-4975-9223-791bd5028769\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") "
Sep 17 11:25:06.307257 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.307068    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-workload-certs\") pod \"0be46ac4-4bc1-4975-9223-791bd5028769\" (UID: \"0be46ac4-4bc1-4975-9223-791bd5028769\") "
Sep 17 11:25:06.307257 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.307184    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-istio-data" (OuterVolumeSpecName: "istio-data") pod "0be46ac4-4bc1-4975-9223-791bd5028769" (UID: "0be46ac4-4bc1-4975-9223-791bd5028769"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:25:06.307632 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.307266    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0be46ac4-4bc1-4975-9223-791bd5028769-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0be46ac4-4bc1-4975-9223-791bd5028769" (UID: "0be46ac4-4bc1-4975-9223-791bd5028769"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:25:06.307632 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.307442    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0be46ac4-4bc1-4975-9223-791bd5028769" (UID: "0be46ac4-4bc1-4975-9223-791bd5028769"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:25:06.307632 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.307484    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be46ac4-4bc1-4975-9223-791bd5028769-istiod-ca-cert\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:25:06.307632 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.307498    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-istio-data\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:25:06.307632 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.307507    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-workload-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:25:06.307632 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.307477    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0be46ac4-4bc1-4975-9223-791bd5028769" (UID: "0be46ac4-4bc1-4975-9223-791bd5028769"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:25:06.309207 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.309184    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be46ac4-4bc1-4975-9223-791bd5028769-istio-token" (OuterVolumeSpecName: "istio-token") pod "0be46ac4-4bc1-4975-9223-791bd5028769" (UID: "0be46ac4-4bc1-4975-9223-791bd5028769"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:25:06.309288 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.309215    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0be46ac4-4bc1-4975-9223-791bd5028769" (UID: "0be46ac4-4bc1-4975-9223-791bd5028769"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 11:25:06.309683 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.309664    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0be46ac4-4bc1-4975-9223-791bd5028769-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0be46ac4-4bc1-4975-9223-791bd5028769" (UID: "0be46ac4-4bc1-4975-9223-791bd5028769"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 11:25:06.309739 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.309686    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be46ac4-4bc1-4975-9223-791bd5028769-kube-api-access-kb66x" (OuterVolumeSpecName: "kube-api-access-kb66x") pod "0be46ac4-4bc1-4975-9223-791bd5028769" (UID: "0be46ac4-4bc1-4975-9223-791bd5028769"). InnerVolumeSpecName "kube-api-access-kb66x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:25:06.408145 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.408121    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be46ac4-4bc1-4975-9223-791bd5028769-istio-token\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:25:06.408145 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.408143    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-credential-socket\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:25:06.408279 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.408152    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be46ac4-4bc1-4975-9223-791bd5028769-istio-podinfo\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:25:06.408279 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.408162    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb66x\" (UniqueName: \"kubernetes.io/projected/0be46ac4-4bc1-4975-9223-791bd5028769-kube-api-access-kb66x\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:25:06.408279 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.408170    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-workload-certs\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:25:06.408279 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:06.408178    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be46ac4-4bc1-4975-9223-791bd5028769-istio-envoy\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:25:07.105915 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:07.105874    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"
Sep 17 11:25:07.106318 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:07.105865    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml" event={"ID":"0be46ac4-4bc1-4975-9223-791bd5028769","Type":"ContainerDied","Data":"31a707be70bf66bd1e269f4b0b717f8de85519faf8ae8f2609c9b9ff9eb341e4"}
Sep 17 11:25:07.106318 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:07.106052    2564 scope.go:117] "RemoveContainer" containerID="b674cbc7d599f381cd8736fd91bda99f4cf58b1b1c71c76b82285f5da8384f41"
Sep 17 11:25:07.126999 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:07.126968    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"]
Sep 17 11:25:07.131513 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:07.131487    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-026f86-openshift-default-5f9b6dfb86-dbjml"]
Sep 17 11:25:07.143019 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:07.142998    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0be46ac4-4bc1-4975-9223-791bd5028769" path="/var/lib/kubelet/pods/0be46ac4-4bc1-4975-9223-791bd5028769/volumes"
Sep 17 11:25:11.827649 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:11.827616    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"]
Sep 17 11:25:11.828158 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:11.827855    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m" podUID="66546020-c5a6-4761-8b6a-2f4cbb4265ad" containerName="payload-processing" containerID="cri-o://28c6bb27d85dd04f8533cb4989dc369934e85e4121a07130a7ea68b1cdf89884" gracePeriod=30
Sep 17 11:25:12.003359 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.003330    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9"]
Sep 17 11:25:12.010791 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.010764    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b8287f-x27c9"]
Sep 17 11:25:12.092131 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.092078    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"
Sep 17 11:25:12.136916 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.136860    2564 generic.go:356] "Generic (PLEG): container finished" podID="66546020-c5a6-4761-8b6a-2f4cbb4265ad" containerID="28c6bb27d85dd04f8533cb4989dc369934e85e4121a07130a7ea68b1cdf89884" exitCode=0
Sep 17 11:25:12.137077 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.136925    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"
Sep 17 11:25:12.137077 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.136942    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m" event={"ID":"66546020-c5a6-4761-8b6a-2f4cbb4265ad","Type":"ContainerDied","Data":"28c6bb27d85dd04f8533cb4989dc369934e85e4121a07130a7ea68b1cdf89884"}
Sep 17 11:25:12.137077 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.136990    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m" event={"ID":"66546020-c5a6-4761-8b6a-2f4cbb4265ad","Type":"ContainerDied","Data":"e952e985b3381b9d9ccb6ab8d7f5d04e184b858d041b156596c7a4aec19fbd89"}
Sep 17 11:25:12.137077 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.137018    2564 scope.go:117] "RemoveContainer" containerID="28c6bb27d85dd04f8533cb4989dc369934e85e4121a07130a7ea68b1cdf89884"
Sep 17 11:25:12.146246 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.146228    2564 scope.go:117] "RemoveContainer" containerID="28c6bb27d85dd04f8533cb4989dc369934e85e4121a07130a7ea68b1cdf89884"
Sep 17 11:25:12.146540 ip-10-0-129-205 kubenswrapper[2564]: E0917 11:25:12.146523    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28c6bb27d85dd04f8533cb4989dc369934e85e4121a07130a7ea68b1cdf89884\": container with ID starting with 28c6bb27d85dd04f8533cb4989dc369934e85e4121a07130a7ea68b1cdf89884 not found: ID does not exist" containerID="28c6bb27d85dd04f8533cb4989dc369934e85e4121a07130a7ea68b1cdf89884"
Sep 17 11:25:12.146585 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.146550    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28c6bb27d85dd04f8533cb4989dc369934e85e4121a07130a7ea68b1cdf89884"} err="failed to get container status \"28c6bb27d85dd04f8533cb4989dc369934e85e4121a07130a7ea68b1cdf89884\": rpc error: code = NotFound desc = could not find container \"28c6bb27d85dd04f8533cb4989dc369934e85e4121a07130a7ea68b1cdf89884\": container with ID starting with 28c6bb27d85dd04f8533cb4989dc369934e85e4121a07130a7ea68b1cdf89884 not found: ID does not exist"
Sep 17 11:25:12.153775 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.153755    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwtsv\" (UniqueName: \"kubernetes.io/projected/66546020-c5a6-4761-8b6a-2f4cbb4265ad-kube-api-access-qwtsv\") pod \"66546020-c5a6-4761-8b6a-2f4cbb4265ad\" (UID: \"66546020-c5a6-4761-8b6a-2f4cbb4265ad\") "
Sep 17 11:25:12.153854 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.153787    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66546020-c5a6-4761-8b6a-2f4cbb4265ad-plugins-config-volume\") pod \"66546020-c5a6-4761-8b6a-2f4cbb4265ad\" (UID: \"66546020-c5a6-4761-8b6a-2f4cbb4265ad\") "
Sep 17 11:25:12.154232 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.154210    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66546020-c5a6-4761-8b6a-2f4cbb4265ad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "66546020-c5a6-4761-8b6a-2f4cbb4265ad" (UID: "66546020-c5a6-4761-8b6a-2f4cbb4265ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 11:25:12.158002 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.157970    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66546020-c5a6-4761-8b6a-2f4cbb4265ad-kube-api-access-qwtsv" (OuterVolumeSpecName: "kube-api-access-qwtsv") pod "66546020-c5a6-4761-8b6a-2f4cbb4265ad" (UID: "66546020-c5a6-4761-8b6a-2f4cbb4265ad"). InnerVolumeSpecName "kube-api-access-qwtsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 11:25:12.255292 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.255259    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwtsv\" (UniqueName: \"kubernetes.io/projected/66546020-c5a6-4761-8b6a-2f4cbb4265ad-kube-api-access-qwtsv\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:25:12.255292 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.255285    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66546020-c5a6-4761-8b6a-2f4cbb4265ad-plugins-config-volume\") on node \"ip-10-0-129-205.ec2.internal\" DevicePath \"\""
Sep 17 11:25:12.484582 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.484546    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"]
Sep 17 11:25:12.494462 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:12.494437    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b8287f-668d744c97-hz99m"]
Sep 17 11:25:13.144880 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:13.144837    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30911f8c-0066-4817-8e8c-d466a073f456" path="/var/lib/kubelet/pods/30911f8c-0066-4817-8e8c-d466a073f456/volumes"
Sep 17 11:25:13.145339 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:13.145239    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66546020-c5a6-4761-8b6a-2f4cbb4265ad" path="/var/lib/kubelet/pods/66546020-c5a6-4761-8b6a-2f4cbb4265ad/volumes"
Sep 17 11:25:16.137067 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:16.137035    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:25:25.594420 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:25.594383    2564 scope.go:117] "RemoveContainer" containerID="49b3a7f73968440cbd5625ff6743bf3c30eda3a91b2835d48000430238d7a68c"
Sep 17 11:25:38.137806 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:25:38.137768    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:26:19.137809 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:26:19.137730    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:26:49.137134 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:26:49.137100    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:27:25.639876 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:27:25.639841    2564 scope.go:117] "RemoveContainer" containerID="985f9ac3cd138fb923aab9798dfd9154b85525595c27dfc0dc969cfb5869a986"
Sep 17 11:27:25.648756 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:27:25.648734    2564 scope.go:117] "RemoveContainer" containerID="12940c3d69f424fef7e40f5fdeb59f74944a5d87c2ebcc840c4bb829b2d4bccf"
Sep 17 11:27:25.656795 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:27:25.656779    2564 scope.go:117] "RemoveContainer" containerID="368a7e0b61d9c05bd2dc7aece91d8b25dced24993946fc6991128e25b44b7375"
Sep 17 11:27:33.137224 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:27:33.137190    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:27:52.136958 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:27:52.136862    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:28:25.683538 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:28:25.683509    2564 scope.go:117] "RemoveContainer" containerID="fce35d29d2ba8e7e64a0731c535993cf3c6fefe141f6f6e92e97ce8a4a939b0a"
Sep 17 11:28:54.136790 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:28:54.136761    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:29:00.136978 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:29:00.136939    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:29:25.706489 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:29:25.706458    2564 scope.go:117] "RemoveContainer" containerID="d2e43bd0251ce0981cdbc72f682e6ad4a44955bf26ccc866609859589529da58"
Sep 17 11:29:25.715121 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:29:25.715099    2564 scope.go:117] "RemoveContainer" containerID="2f40ea655079dbca81a464cb45b922022443cf4df83592f66d0d5c2f91b2f07e"
Sep 17 11:29:56.137502 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:29:56.137469    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:30:00.156428 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.156393    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29827410-hfkx9"]
Sep 17 11:30:00.156832 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.156731    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0be46ac4-4bc1-4975-9223-791bd5028769" containerName="istio-proxy"
Sep 17 11:30:00.156832 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.156742    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0be46ac4-4bc1-4975-9223-791bd5028769" containerName="istio-proxy"
Sep 17 11:30:00.156832 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.156753    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0be46ac4-4bc1-4975-9223-791bd5028769" containerName="istio-proxy"
Sep 17 11:30:00.156832 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.156758    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0be46ac4-4bc1-4975-9223-791bd5028769" containerName="istio-proxy"
Sep 17 11:30:00.156832 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.156769    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30911f8c-0066-4817-8e8c-d466a073f456" containerName="revoke-keys"
Sep 17 11:30:00.156832 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.156775    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="30911f8c-0066-4817-8e8c-d466a073f456" containerName="revoke-keys"
Sep 17 11:30:00.156832 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.156784    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66546020-c5a6-4761-8b6a-2f4cbb4265ad" containerName="payload-processing"
Sep 17 11:30:00.156832 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.156790    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="66546020-c5a6-4761-8b6a-2f4cbb4265ad" containerName="payload-processing"
Sep 17 11:30:00.157118 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.156843    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0be46ac4-4bc1-4975-9223-791bd5028769" containerName="istio-proxy"
Sep 17 11:30:00.157118 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.156850    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0be46ac4-4bc1-4975-9223-791bd5028769" containerName="istio-proxy"
Sep 17 11:30:00.157118 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.156858    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="66546020-c5a6-4761-8b6a-2f4cbb4265ad" containerName="payload-processing"
Sep 17 11:30:00.157118 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.156864    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="30911f8c-0066-4817-8e8c-d466a073f456" containerName="revoke-keys"
Sep 17 11:30:00.159617 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.159601    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827410-hfkx9"
Sep 17 11:30:00.161722 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.161707    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pnptl\""
Sep 17 11:30:00.166767 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.166740    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29827410-hfkx9"]
Sep 17 11:30:00.170196 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.170175    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827410-hfkx9"
Sep 17 11:30:00.446539 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.446348    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29827410-hfkx9"]
Sep 17 11:30:00.449188 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:30:00.449158    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode632fab0_7347_434d_a857_4c64dcfb4873.slice/crio-d4ae4759640042457bead950e19aad102ce77891b5a0e6085cd9c3e8b8396fa2 WatchSource:0}: Error finding container d4ae4759640042457bead950e19aad102ce77891b5a0e6085cd9c3e8b8396fa2: Status 404 returned error can't find the container with id d4ae4759640042457bead950e19aad102ce77891b5a0e6085cd9c3e8b8396fa2
Sep 17 11:30:00.714185 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.714152    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827410-hfkx9" event={"ID":"e632fab0-7347-434d-a857-4c64dcfb4873","Type":"ContainerStarted","Data":"a89e93f02af1379476a2d90d5b542a5ad19877ba5876ffe023b938b725857aba"}
Sep 17 11:30:00.714316 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.714234    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827410-hfkx9" event={"ID":"e632fab0-7347-434d-a857-4c64dcfb4873","Type":"ContainerStarted","Data":"d4ae4759640042457bead950e19aad102ce77891b5a0e6085cd9c3e8b8396fa2"}
Sep 17 11:30:00.730376 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:00.730332    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827410-hfkx9" podStartSLOduration=0.730316944 podStartE2EDuration="730.316944ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 11:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:30:00.727602418 +0000 UTC m=+2196.043965573" watchObservedRunningTime="2026-09-17 11:30:00.730316944 +0000 UTC m=+2196.046680119"
Sep 17 11:30:01.720783 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:01.720691    2564 generic.go:356] "Generic (PLEG): container finished" podID="e632fab0-7347-434d-a857-4c64dcfb4873" containerID="a89e93f02af1379476a2d90d5b542a5ad19877ba5876ffe023b938b725857aba" exitCode=0
Sep 17 11:30:01.721230 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:01.720773    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827410-hfkx9" event={"ID":"e632fab0-7347-434d-a857-4c64dcfb4873","Type":"ContainerDied","Data":"a89e93f02af1379476a2d90d5b542a5ad19877ba5876ffe023b938b725857aba"}
Sep 17 11:30:02.850662 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:02.850637    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827410-hfkx9"
Sep 17 11:30:03.732119 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:03.732087    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827410-hfkx9"
Sep 17 11:30:03.732295 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:03.732121    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29827410-hfkx9" event={"ID":"e632fab0-7347-434d-a857-4c64dcfb4873","Type":"ContainerDied","Data":"d4ae4759640042457bead950e19aad102ce77891b5a0e6085cd9c3e8b8396fa2"}
Sep 17 11:30:03.732295 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:03.732168    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4ae4759640042457bead950e19aad102ce77891b5a0e6085cd9c3e8b8396fa2"
Sep 17 11:30:30.137621 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:30.137587    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:30:57.137427 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:30:57.137341    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:31:25.749049 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:31:25.749021    2564 scope.go:117] "RemoveContainer" containerID="cb0351ebeba2bc7f61c010f291e54271129c5df8646d2d62fe50ea4d6ed1154e"
Sep 17 11:31:25.758046 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:31:25.758027    2564 scope.go:117] "RemoveContainer" containerID="81685373ed487d106a24d07ebdbb6c796cfe07b8ae836394eaed1cca1e2f51e7"
Sep 17 11:31:25.765866 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:31:25.765849    2564 scope.go:117] "RemoveContainer" containerID="77d465e79a719bba766a57c40715562464450123a9b120a41fe68f2e2f992742"
Sep 17 11:31:46.137185 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:31:46.137151    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:32:06.137707 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:32:06.137673    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:32:48.137093 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:32:48.137059    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:33:19.137299 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:33:19.137256    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-rl88q" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:34:02.166422 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:02.166336    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ddcnf_909dc7e0-fcc5-4736-8874-8d5206f3f741/global-pull-secret-syncer/0.log"
Sep 17 11:34:02.317357 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:02.317328    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fdnxp_3d1fe18b-e374-4453-a560-3aa3245d100b/konnectivity-agent/0.log"
Sep 17 11:34:02.373495 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:02.373461    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-205.ec2.internal_e413a23ab9533a4b31565bf83d28336b/haproxy/0.log"
Sep 17 11:34:05.141446 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:05.141406    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-6cd864bc95-4l7wg" secret="" err="secret \"payload-processing-dockercfg-rklfc\" not found"
Sep 17 11:34:10.213664 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:10.213627    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-9s4c2_51364bda-4f67-4195-b557-43e2fb6726ca/kube-state-metrics/0.log"
Sep 17 11:34:10.243866 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:10.243829    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-9s4c2_51364bda-4f67-4195-b557-43e2fb6726ca/kube-rbac-proxy-main/0.log"
Sep 17 11:34:10.289492 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:10.289467    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-9s4c2_51364bda-4f67-4195-b557-43e2fb6726ca/kube-rbac-proxy-self/0.log"
Sep 17 11:34:10.322920 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:10.322887    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-67dc9f7ff4-8r5zd_deaaeeb5-cb5e-45fa-a284-07052b47268e/metrics-server/0.log"
Sep 17 11:34:10.485049 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:10.485026    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r9tvg_c6048ca1-2e34-4de8-b8bd-4661504dc4e4/node-exporter/0.log"
Sep 17 11:34:10.515262 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:10.515239    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r9tvg_c6048ca1-2e34-4de8-b8bd-4661504dc4e4/kube-rbac-proxy/0.log"
Sep 17 11:34:10.548083 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:10.548064    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r9tvg_c6048ca1-2e34-4de8-b8bd-4661504dc4e4/init-textfile/0.log"
Sep 17 11:34:10.682339 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:10.682311    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-24xnl_b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c/kube-rbac-proxy-main/0.log"
Sep 17 11:34:10.710521 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:10.710500    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-24xnl_b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c/kube-rbac-proxy-self/0.log"
Sep 17 11:34:10.739354 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:10.739245    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-24xnl_b9afe1f0-74e7-4ea4-b8dc-e872d34dcc0c/openshift-state-metrics/0.log"
Sep 17 11:34:11.002585 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:11.002508    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-vq86r_8ca4015e-76d6-45ce-89ba-5dacd3f6aac4/prometheus-operator/0.log"
Sep 17 11:34:11.026284 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:11.026264    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-vq86r_8ca4015e-76d6-45ce-89ba-5dacd3f6aac4/kube-rbac-proxy/0.log"
Sep 17 11:34:11.939492 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:11.939465    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"]
Sep 17 11:34:11.939885 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:11.939794    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e632fab0-7347-434d-a857-4c64dcfb4873" containerName="cleanup"
Sep 17 11:34:11.939885 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:11.939806    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e632fab0-7347-434d-a857-4c64dcfb4873" containerName="cleanup"
Sep 17 11:34:11.940009 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:11.939891    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e632fab0-7347-434d-a857-4c64dcfb4873" containerName="cleanup"
Sep 17 11:34:11.942928 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:11.942886    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:11.945153 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:11.945134    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tmn9t\"/\"openshift-service-ca.crt\""
Sep 17 11:34:11.945703 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:11.945682    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tmn9t\"/\"default-dockercfg-scghf\""
Sep 17 11:34:11.945863 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:11.945700    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tmn9t\"/\"kube-root-ca.crt\""
Sep 17 11:34:11.952382 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:11.952361    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"]
Sep 17 11:34:12.037759 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.037736    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e495e21e-a7ec-467a-bbea-170eb68157af-lib-modules\") pod \"perf-node-gather-daemonset-r5rdq\" (UID: \"e495e21e-a7ec-467a-bbea-170eb68157af\") " pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:12.037935 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.037774    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mnk8\" (UniqueName: \"kubernetes.io/projected/e495e21e-a7ec-467a-bbea-170eb68157af-kube-api-access-4mnk8\") pod \"perf-node-gather-daemonset-r5rdq\" (UID: \"e495e21e-a7ec-467a-bbea-170eb68157af\") " pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:12.037935 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.037809    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e495e21e-a7ec-467a-bbea-170eb68157af-sys\") pod \"perf-node-gather-daemonset-r5rdq\" (UID: \"e495e21e-a7ec-467a-bbea-170eb68157af\") " pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:12.037935 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.037886    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e495e21e-a7ec-467a-bbea-170eb68157af-podres\") pod \"perf-node-gather-daemonset-r5rdq\" (UID: \"e495e21e-a7ec-467a-bbea-170eb68157af\") " pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:12.038064 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.037948    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e495e21e-a7ec-467a-bbea-170eb68157af-proc\") pod \"perf-node-gather-daemonset-r5rdq\" (UID: \"e495e21e-a7ec-467a-bbea-170eb68157af\") " pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:12.139264 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.139228    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e495e21e-a7ec-467a-bbea-170eb68157af-lib-modules\") pod \"perf-node-gather-daemonset-r5rdq\" (UID: \"e495e21e-a7ec-467a-bbea-170eb68157af\") " pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:12.139420 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.139284    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mnk8\" (UniqueName: \"kubernetes.io/projected/e495e21e-a7ec-467a-bbea-170eb68157af-kube-api-access-4mnk8\") pod \"perf-node-gather-daemonset-r5rdq\" (UID: \"e495e21e-a7ec-467a-bbea-170eb68157af\") " pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:12.139420 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.139341    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e495e21e-a7ec-467a-bbea-170eb68157af-sys\") pod \"perf-node-gather-daemonset-r5rdq\" (UID: \"e495e21e-a7ec-467a-bbea-170eb68157af\") " pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:12.139420 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.139378    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e495e21e-a7ec-467a-bbea-170eb68157af-podres\") pod \"perf-node-gather-daemonset-r5rdq\" (UID: \"e495e21e-a7ec-467a-bbea-170eb68157af\") " pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:12.139420 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.139403    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e495e21e-a7ec-467a-bbea-170eb68157af-lib-modules\") pod \"perf-node-gather-daemonset-r5rdq\" (UID: \"e495e21e-a7ec-467a-bbea-170eb68157af\") " pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:12.139613 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.139441    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e495e21e-a7ec-467a-bbea-170eb68157af-sys\") pod \"perf-node-gather-daemonset-r5rdq\" (UID: \"e495e21e-a7ec-467a-bbea-170eb68157af\") " pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:12.139613 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.139453    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e495e21e-a7ec-467a-bbea-170eb68157af-proc\") pod \"perf-node-gather-daemonset-r5rdq\" (UID: \"e495e21e-a7ec-467a-bbea-170eb68157af\") " pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:12.139613 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.139410    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e495e21e-a7ec-467a-bbea-170eb68157af-proc\") pod \"perf-node-gather-daemonset-r5rdq\" (UID: \"e495e21e-a7ec-467a-bbea-170eb68157af\") " pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:12.139613 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.139506    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e495e21e-a7ec-467a-bbea-170eb68157af-podres\") pod \"perf-node-gather-daemonset-r5rdq\" (UID: \"e495e21e-a7ec-467a-bbea-170eb68157af\") " pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:12.150026 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.150005    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mnk8\" (UniqueName: \"kubernetes.io/projected/e495e21e-a7ec-467a-bbea-170eb68157af-kube-api-access-4mnk8\") pod \"perf-node-gather-daemonset-r5rdq\" (UID: \"e495e21e-a7ec-467a-bbea-170eb68157af\") " pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:12.253309 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.253290    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:12.535526 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.535499    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"]
Sep 17 11:34:12.537953 ip-10-0-129-205 kubenswrapper[2564]: W0917 11:34:12.537920    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode495e21e_a7ec_467a_bbea_170eb68157af.slice/crio-2b9e90e601bd423fb632b4179bfc72e0809a275f6d763c5bbcf6b550413bf1f2 WatchSource:0}: Error finding container 2b9e90e601bd423fb632b4179bfc72e0809a275f6d763c5bbcf6b550413bf1f2: Status 404 returned error can't find the container with id 2b9e90e601bd423fb632b4179bfc72e0809a275f6d763c5bbcf6b550413bf1f2
Sep 17 11:34:12.560298 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:12.560275    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-56ptm_9c0a4d3d-f882-4df0-abaf-6e0cfe5ec065/networking-console-plugin/0.log"
Sep 17 11:34:13.042962 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:13.042918    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq" event={"ID":"e495e21e-a7ec-467a-bbea-170eb68157af","Type":"ContainerStarted","Data":"8155d6f6bf0112dd29eefbf556b62832f09240f308652aa082c033e319702b4e"}
Sep 17 11:34:13.043392 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:13.042981    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq" event={"ID":"e495e21e-a7ec-467a-bbea-170eb68157af","Type":"ContainerStarted","Data":"2b9e90e601bd423fb632b4179bfc72e0809a275f6d763c5bbcf6b550413bf1f2"}
Sep 17 11:34:13.043392 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:13.043036    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:13.070415 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:13.070360    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq" podStartSLOduration=2.070345695 podStartE2EDuration="2.070345695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 11:34:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 11:34:13.069028436 +0000 UTC m=+2448.385391605" watchObservedRunningTime="2026-09-17 11:34:13.070345695 +0000 UTC m=+2448.386708849"
Sep 17 11:34:15.894110 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:15.894079    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f466l_a3b2ce07-6399-4354-8cdd-be6af04d46a8/dns/0.log"
Sep 17 11:34:15.981977 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:15.981954    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f466l_a3b2ce07-6399-4354-8cdd-be6af04d46a8/kube-rbac-proxy/0.log"
Sep 17 11:34:16.338153 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:16.338124    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-phs4b_a02f264e-0ec3-4ea3-b2a8-aca37526f1c3/dns-node-resolver/0.log"
Sep 17 11:34:17.155782 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:17.155753    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-86d9d67866-242j2_57c6e2bf-09ef-4558-99e6-a4befe6d8505/registry/0.log"
Sep 17 11:34:17.335251 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:17.335221    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rdz5r_f7775776-e097-4507-9a6a-5d21c14ebe07/node-ca/0.log"
Sep 17 11:34:18.532877 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:18.532835    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd_0e730ef5-89da-4980-a9a1-77060acf61eb/istio-proxy/0.log"
Sep 17 11:34:19.061449 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:19.061417    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tmn9t/perf-node-gather-daemonset-r5rdq"
Sep 17 11:34:19.169085 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:19.169059    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-dqjkh_156dfbf0-52a9-4789-baa4-9bf47c07c620/discovery/0.log"
Sep 17 11:34:19.284493 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:19.284457    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-6cd864bc95-4l7wg_3130c736-c680-4e0a-ada7-189fd19b3541/payload-pre-processing/0.log"
Sep 17 11:34:19.308389 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:19.308364    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-rl88q_995b3ba4-d76f-4b47-8194-940111359da7/payload-processing/0.log"
Sep 17 11:34:20.302532 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:20.302500    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vrd77_80044c26-9705-435f-a719-4a0e57ac5c75/serve-healthcheck-canary/0.log"
Sep 17 11:34:20.883594 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:20.883564    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nwtnk_79ee78ea-aacc-4ecb-a534-f4a8cfd4e349/kube-rbac-proxy/0.log"
Sep 17 11:34:20.908484 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:20.908454    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nwtnk_79ee78ea-aacc-4ecb-a534-f4a8cfd4e349/exporter/0.log"
Sep 17 11:34:20.932473 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:20.932451    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nwtnk_79ee78ea-aacc-4ecb-a534-f4a8cfd4e349/extractor/0.log"
Sep 17 11:34:22.728986 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:22.728949    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd_0e730ef5-89da-4980-a9a1-77060acf61eb/istio-proxy/0.log"
Sep 17 11:34:23.358523 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:23.358493    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-dqjkh_156dfbf0-52a9-4789-baa4-9bf47c07c620/discovery/0.log"
Sep 17 11:34:23.451496 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:23.451423    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-6cd864bc95-4l7wg_3130c736-c680-4e0a-ada7-189fd19b3541/payload-pre-processing/0.log"
Sep 17 11:34:23.469422 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:23.469391    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-rl88q_995b3ba4-d76f-4b47-8194-940111359da7/payload-processing/0.log"
Sep 17 11:34:23.515748 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:23.515720    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6ccb46fcc-7zd6v_e1599327-332a-4f62-908b-04c8edaed19e/manager/0.log"
Sep 17 11:34:23.818044 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:23.818012    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7kqvf_2b5cd969-4bc5-4f8e-89e3-1e36d3598b36/server/0.log"
Sep 17 11:34:23.953205 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:23.953172    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mnwq4_d1c78d62-da7a-4cd9-98bd-377a059e6613/manager/0.log"
Sep 17 11:34:24.029470 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:24.029435    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c54c4bd97-bw97f_8109edee-2793-485d-8998-9b4dc165396f/manager/0.log"
Sep 17 11:34:24.047593 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:24.047570    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c54c4bd97-bw97f_8109edee-2793-485d-8998-9b4dc165396f/copy-manifests/0.log"
Sep 17 11:34:24.963764 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:24.963735    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-c2txd_0e730ef5-89da-4980-a9a1-77060acf61eb/istio-proxy/0.log"
Sep 17 11:34:25.592643 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:25.592609    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-dqjkh_156dfbf0-52a9-4789-baa4-9bf47c07c620/discovery/0.log"
Sep 17 11:34:25.669914 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:25.669872    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7bb4f45b46-sg2qj_4ac624b0-4480-4a91-a803-a1dd59b1ada2/manager/0.log"
Sep 17 11:34:25.776887 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:25.776856    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-6cd864bc95-4l7wg_3130c736-c680-4e0a-ada7-189fd19b3541/payload-pre-processing/0.log"
Sep 17 11:34:25.800861 ip-10-0-129-205 kubenswrapper[2564]: I0917 11:34:25.800833    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-rl88q_995b3ba4-d76f-4b47-8194-940111359da7/payload-processing/0.log"