Sep 07 11:45:20.433789 ip-10-0-128-68 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 07 11:45:20.433799 ip-10-0-128-68 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 07 11:45:20.433809 ip-10-0-128-68 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 07 11:45:20.434050 ip-10-0-128-68 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 07 11:45:30.622095 ip-10-0-128-68 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 07 11:45:30.622112 ip-10-0-128-68 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot eb1d4413466b4ae7bc6bff51d3cfb402 --
Sep 07 11:47:25.381515 ip-10-0-128-68 systemd[1]: Starting Kubernetes Kubelet...
Sep 07 11:47:25.802744 ip-10-0-128-68 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 11:47:25.802744 ip-10-0-128-68 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 07 11:47:25.802744 ip-10-0-128-68 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 11:47:25.802744 ip-10-0-128-68 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 07 11:47:25.802744 ip-10-0-128-68 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 11:47:25.804428 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.804340    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 07 11:47:25.809045 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809027    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:47:25.809045 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809044    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:47:25.809045 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809048    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809052    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809055    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809058    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809061    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809064    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809066    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809069    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809072    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809074    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809084    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809087    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809089    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809092    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809095    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809099    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809103    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809106    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809108    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:47:25.809141 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809111    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809114    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809116    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809119    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809124    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809128    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809131    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809134    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809137    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809140    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809142    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809145    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809148    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809151    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809154    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809157    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809159    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809162    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809164    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:47:25.809622 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809167    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809169    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809172    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809174    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809177    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809180    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809183    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809185    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809188    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809190    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809193    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809208    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809210    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809213    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809216    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809220    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809224    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809227    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809230    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:47:25.810080 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809233    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809236    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809239    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809242    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809245    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809247    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809250    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809252    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809255    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809258    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809260    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809263    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809267    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809270    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809273    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809275    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809278    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809280    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809283    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809285    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:47:25.810556 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809288    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809291    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809293    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809296    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809299    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809301    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809304    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809306    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809310    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809313    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809316    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809319    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809321    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809324    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809326    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809329    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809332    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809334    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809337    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.809339    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:47:25.811026 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810925    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810934    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810937    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810940    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810943    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810947    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810950    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810954    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810956    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810959    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810962    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810965    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810968    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810970    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810973    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810976    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810979    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810981    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810985    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810989    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:47:25.811507 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810993    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810996    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.810999    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811002    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811005    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811007    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811010    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811012    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811015    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811017    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811020    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811023    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811026    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811029    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811031    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811034    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811037    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811040    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811043    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811045    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:47:25.811997 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811048    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811051    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811053    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811055    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811058    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811060    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811063    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811065    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811068    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811070    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811073    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811075    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811079    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811081    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811085    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811090    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811093    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811096    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811099    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811101    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:47:25.812498 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811104    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811107    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811110    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811113    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811115    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811118    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811120    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811123    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811125    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811127    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811130    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811133    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811136    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811138    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811141    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811143    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811146    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811148    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811151    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:47:25.812980 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811153    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811156    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811158    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811161    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811163    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811166    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811170    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811174    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811176    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811179    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811182    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811184    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811187    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811189    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811192    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811207    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811210    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811213    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811215    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:47:25.813478 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.811218    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812052    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812058    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812061    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812064    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812067    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812071    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812073    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812076    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812078    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812081    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812083    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812086    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812089    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812091    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812094    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812096    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812099    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812102    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812105    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:47:25.814030 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812107    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812110    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812112    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812115    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812117    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812120    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812123    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812126    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812128    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812130    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812133    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812136    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812139    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812142    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812144    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812147    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812149    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812153    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812156    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:47:25.814545 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812158    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812162    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812166    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812168    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812171    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812173    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812176    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812179    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812182    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812187    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812189    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812193    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812211    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812215    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812218    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812222    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812224    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812227    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:47:25.815028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812230    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812233    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812236    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812238    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812241    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812243    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812246    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812249    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812251    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812253    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812256    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812260    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812263    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812266    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812269    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812272    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812275    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812278    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812281    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:47:25.815470 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812284    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812286    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812289    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812292    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812296    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812299    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812302    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812304    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812307    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812310    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812313    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812316    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812319    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812322    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812324    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812327    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812329    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812332    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812334    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812337    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:47:25.815924 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812340    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812342    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812345    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.812347    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812411    2580 flags.go:64] FLAG: --address="0.0.0.0"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812448    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812457    2580 flags.go:64] FLAG: --anonymous-auth="true"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812462    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812468    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812471    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812475    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812480    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812483    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812486    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812489    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812493    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812497    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812501    2580 flags.go:64] FLAG: --cgroup-root=""
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812505    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812508    2580 flags.go:64] FLAG: --client-ca-file=""
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812511    2580 flags.go:64] FLAG: --cloud-provider="external"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812514    2580 flags.go:64] FLAG: --cluster-dns="[]"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812519    2580 flags.go:64] FLAG: --cluster-domain=""
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812522    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 07 11:47:25.816451 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812525    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812528    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812532    2580 flags.go:64] FLAG: --container-log-max-files="5"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812536    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812539    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812542    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812545    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812548    2580 flags.go:64] FLAG: --contention-profiling="false"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812551    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812554    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812557    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812560    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812569    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812572    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812575    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812578    2580 flags.go:64] FLAG: --enable-load-reader="false"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812581    2580 flags.go:64] FLAG: --enable-server="true"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812590    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812595    2580 flags.go:64] FLAG: --event-burst="100"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812598    2580 flags.go:64] FLAG: --event-qps="50"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812601    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812604    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812607    2580 flags.go:64] FLAG: --eviction-hard=""
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812614    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 07 11:47:25.817035 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812618    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812622    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812625    2580 flags.go:64] FLAG: --eviction-soft=""
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812628    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812631    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812634    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812637    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812640    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812643    2580 flags.go:64] FLAG: --fail-swap-on="true"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812646    2580 flags.go:64] FLAG: --feature-gates=""
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812650    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812653    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812656    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812659    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812662    2580 flags.go:64] FLAG: --healthz-port="10248"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812665    2580 flags.go:64] FLAG: --help="false"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812668    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-128-68.ec2.internal"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812671    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812674    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812677    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812681    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812684    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812687    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 07 11:47:25.817617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812690    2580 flags.go:64] FLAG: --image-service-endpoint=""
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812693    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812696    2580 flags.go:64] FLAG: --kube-api-burst="100"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812699    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812703    2580 flags.go:64] FLAG: --kube-api-qps="50"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812706    2580 flags.go:64] FLAG: --kube-reserved=""
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812709    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812711    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812714    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812718    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812721    2580 flags.go:64] FLAG: --lock-file=""
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812724    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812727    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812730    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812736    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812739    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812741    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812744    2580 flags.go:64] FLAG: --logging-format="text"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812747    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812750    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812754    2580 flags.go:64] FLAG: --manifest-url=""
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812756    2580 flags.go:64] FLAG: --manifest-url-header=""
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812761    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812764    2580 flags.go:64] FLAG: --max-open-files="1000000"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812768    2580 flags.go:64] FLAG: --max-pods="110"
Sep 07 11:47:25.818194 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812771    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812774    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812777    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812782    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812785    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812788    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812791    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812798    2580 flags.go:64] FLAG: --node-status-max-images="50"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812801    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812804    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812807    2580 flags.go:64] FLAG: --pod-cidr=""
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812810    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812815    2580 flags.go:64] FLAG: --pod-manifest-path=""
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812818    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812821    2580 flags.go:64] FLAG: --pods-per-core="0"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812825    2580 flags.go:64] FLAG: --port="10250"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812829    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812832    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02b6f103d9747907c"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812835    2580 flags.go:64] FLAG: --qos-reserved=""
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812838    2580 flags.go:64] FLAG: --read-only-port="10255"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812841    2580 flags.go:64] FLAG: --register-node="true"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812844    2580 flags.go:64] FLAG: --register-with-taints=""
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812848    2580 flags.go:64] FLAG: --registry-burst="10"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812850    2580 flags.go:64] FLAG: --registry-qps="5"
Sep 07 11:47:25.818816 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812853    2580 flags.go:64] FLAG: --reserved-cpus=""
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812856    2580 flags.go:64] FLAG: --reserved-memory=""
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812860    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812863    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812866    2580 flags.go:64] FLAG: --rotate-certificates="false"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812870    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812873    2580 flags.go:64] FLAG: --runonce="false"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812876    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812879    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812882    2580 flags.go:64] FLAG: --seccomp-default="false"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812885    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812887    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812892    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812895    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812898    2580 flags.go:64] FLAG: --storage-driver-password="root"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812901    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812903    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812906    2580 flags.go:64] FLAG: --storage-driver-user="root"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812909    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812912    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812916    2580 flags.go:64] FLAG: --system-cgroups=""
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812919    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812924    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812927    2580 flags.go:64] FLAG: --tls-cert-file=""
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812931    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 07 11:47:25.819459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812936    2580 flags.go:64] FLAG: --tls-min-version=""
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812938    2580 flags.go:64] FLAG: --tls-private-key-file=""
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812941    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812944    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812947    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812950    2580 flags.go:64] FLAG: --v="2"
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812954    2580 flags.go:64] FLAG: --version="false"
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812958    2580 flags.go:64] FLAG: --vmodule=""
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812962    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.812965    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813076    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813081    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813084    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813087    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813089    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813092    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813094    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813097    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813099    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813103    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813106    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813109    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:47:25.820105 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813111    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813114    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813116    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813119    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813122    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813124    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813127    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813130    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813133    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813136    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813139    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813141    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813144    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813146    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813149    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813152    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813154    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813157    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813159    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813162    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:47:25.820956 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813165    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813167    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813170    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813172    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813175    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813177    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813180    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813182    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813185    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813188    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813191    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813208    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813213    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813217    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813222    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813226    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813229    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813232    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813235    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813239    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:47:25.821724 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813242    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813245    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813248    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813251    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813254    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813256    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813259    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813262    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813264    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813267    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813269    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813272    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813274    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813277    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813279    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813282    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813292    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813295    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813298    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813300    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:47:25.822306 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813303    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813307    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813310    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813312    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813315    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813318    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813320    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813323    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813326    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813328    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813331    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813334    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813338    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813341    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813345    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813348    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813351    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813355    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813358    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:47:25.822793 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813361    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:47:25.823283 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813363    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:47:25.823283 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813366    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:47:25.823283 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813368    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:47:25.823283 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813371    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:47:25.823283 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813373    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:47:25.823283 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813376    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:47:25.823283 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.813378    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:47:25.823283 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.813907    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 11:47:25.823283 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.822088    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 07 11:47:25.823283 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.822103    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 07 11:47:25.823283 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.822751    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 07 11:47:25.823283 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.822759    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 07 11:47:25.823283 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822813    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:47:25.823283 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822819    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:47:25.823283 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822822    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822825    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822828    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822831    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822833    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822836    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822839    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822841    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822844    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822848    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822852    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822855    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822857    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822860    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822863    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822866    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822869    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822871    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822877    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:47:25.823659 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822880    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822883    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822885    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822888    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822891    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822893    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822896    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822898    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822902    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822905    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822907    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822910    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822913    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822916    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822919    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822921    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822924    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822927    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822929    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822932    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:47:25.824112 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822935    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822937    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822941    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822944    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822947    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822950    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822952    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822955    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822957    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822960    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822962    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822965    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822967    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822970    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822973    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822975    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822978    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822980    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822983    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822985    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:47:25.824642 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822989    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822991    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822994    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822997    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.822999    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823002    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823004    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823007    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823009    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823012    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823014    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823017    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823019    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823022    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823029    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823032    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823035    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823037    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823041    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823043    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:47:25.825132 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823046    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823049    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823051    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823054    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823056    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823059    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823061    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823064    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823066    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823069    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823071    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823074    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823079    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823083    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823086    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823089    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823092    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:47:25.825638 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823095    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:47:25.826131 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.823100    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 11:47:25.826131 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823238    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 11:47:25.826131 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823244    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 11:47:25.826131 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823247    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 11:47:25.826131 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823251    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 11:47:25.826131 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823255    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 11:47:25.826131 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823259    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 11:47:25.826131 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823262    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 11:47:25.826131 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823265    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 11:47:25.826131 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823267    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 11:47:25.826131 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823270    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 11:47:25.826131 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823272    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 11:47:25.826131 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823275    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 11:47:25.826131 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823278    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 11:47:25.826131 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823281    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823283    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823286    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823289    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823291    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823295    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823299    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823302    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823305    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823308    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823310    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823313    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823316    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823319    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823322    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823324    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823326    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823329    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823332    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 11:47:25.826517 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823334    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823337    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823339    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823341    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823344    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823346    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823349    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823351    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823354    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823356    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823358    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823361    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823363    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823366    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823369    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823371    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823374    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823376    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823380    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823383    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 11:47:25.827077 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823386    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823389    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823391    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823394    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823396    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823399    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823402    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823404    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823407    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823410    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823412    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823414    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823417    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823419    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823422    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823424    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823427    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823429    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823432    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823434    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 11:47:25.827605 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823437    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823439    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823442    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823444    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823447    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823449    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823452    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823455    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823458    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823460    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823463    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823465    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823467    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823470    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823472    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823475    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823478    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823480    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823483    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823485    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 11:47:25.828139 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823488    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 11:47:25.828820 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823490    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 11:47:25.828820 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823493    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 11:47:25.828820 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823495    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 11:47:25.828820 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823498    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 11:47:25.828820 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823501    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 11:47:25.828820 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:25.823503    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 11:47:25.828820 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.823508    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 11:47:25.828820 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.823628    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 07 11:47:25.828820 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.825690    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 07 11:47:25.828820 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.826543    2580 server.go:1019] "Starting client certificate rotation"
Sep 07 11:47:25.828820 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.826657    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 07 11:47:25.828820 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.826707    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 07 11:47:25.829739 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.829723    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 07 11:47:25.830856 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.830836    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 07 11:47:25.843711 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.843693    2580 log.go:25] "Validated CRI v1 runtime API"
Sep 07 11:47:25.847630 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.847615    2580 log.go:25] "Validated CRI v1 image API"
Sep 07 11:47:25.848506 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.848492    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 07 11:47:25.851896 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.851876    2580 fs.go:135] Filesystem UUIDs: map[04c49e19-8ca5-419f-a42c-49634ce84f3f:/dev/nvme0n1p4 06612977-e128-48e2-9229-b6c072452b29:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 07 11:47:25.851942 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.851895    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 07 11:47:25.857329 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.857160    2580 manager.go:216] Machine: {Timestamp:2026-09-07 11:47:25.855834575 +0000 UTC m=+0.351723989 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099491 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec262754f9f7139f226473f17b415c86 SystemUUID:ec262754-f9f7-139f-2264-73f17b415c86 BootID:eb1d4413-466b-4ae7-bc6b-ff51d3cfb402 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ae:ef:b5:20:a3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ae:ef:b5:20:a3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:86:ad:b9:da:2c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 07 11:47:25.857329 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.857326    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 07 11:47:25.857468 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.857457    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 07 11:47:25.859096 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.859076    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 07 11:47:25.859096 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.859081    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 07 11:47:25.859336 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.859112    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-68.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 07 11:47:25.859942 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.859932    2580 topology_manager.go:138] "Creating topology manager with none policy"
Sep 07 11:47:25.859972 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.859946    2580 container_manager_linux.go:309] "Creating device plugin manager"
Sep 07 11:47:25.859972 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.859954    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 07 11:47:25.860808 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.860798    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 07 11:47:25.862050 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.862038    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 07 11:47:25.865002 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.864991    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 07 11:47:25.865127 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.865118    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 07 11:47:25.865165 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.865145    2580 kubelet.go:486] "Attempting to sync node with API server"
Sep 07 11:47:25.865165 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.865158    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 07 11:47:25.865247 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.865175    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 07 11:47:25.865247 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.865185    2580 kubelet.go:398] "Adding apiserver pod source"
Sep 07 11:47:25.865247 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.865193    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 07 11:47:25.865720 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.865703    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 07 11:47:25.866522 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.866456    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 07 11:47:25.869576 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.869559    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 07 11:47:25.871784 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.871767    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 07 11:47:25.871861 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.871806    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 07 11:47:25.873066 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.873053    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 07 11:47:25.873128 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.873076    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 07 11:47:25.873128 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.873086    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 07 11:47:25.873128 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.873093    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 07 11:47:25.873128 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.873107    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 07 11:47:25.873128 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.873116    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 07 11:47:25.873355 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.873136    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 07 11:47:25.873355 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.873145    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 07 11:47:25.873355 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.873155    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 07 11:47:25.873355 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.873167    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 07 11:47:25.873355 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.873179    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 07 11:47:25.873355 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.873188    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 07 11:47:25.873355 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.873239    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 07 11:47:25.873355 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.873250    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 07 11:47:25.874780 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:25.874745    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 07 11:47:25.874922 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.874902    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-68.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 07 11:47:25.874981 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:25.874956    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-68.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 07 11:47:25.877248 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.877234    2580 server.go:1268] "Started kubelet"
Sep 07 11:47:25.877331 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.877276    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 07 11:47:25.877482 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.877428    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 07 11:47:25.877526 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.877499    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 07 11:47:25.878121 ip-10-0-128-68 systemd[1]: Started Kubernetes Kubelet.
Sep 07 11:47:25.879325 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.879264    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 07 11:47:25.880325 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.880312    2580 server.go:310] "Adding debug handlers to kubelet server"
Sep 07 11:47:25.881849 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:25.881834    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 07 11:47:25.883100 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.883085    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 07 11:47:25.883171 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.883128    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 07 11:47:25.883229 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:25.883183    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-68.ec2.internal\" not found"
Sep 07 11:47:25.883407 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:25.882082    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-68.ec2.internal.18d3076df73fa0c3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-68.ec2.internal,UID:ip-10-0-128-68.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-68.ec2.internal,},FirstTimestamp:2026-09-07 11:47:25.877190851 +0000 UTC m=+0.373080265,LastTimestamp:2026-09-07 11:47:25.877190851 +0000 UTC m=+0.373080265,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-68.ec2.internal,}"
Sep 07 11:47:25.883634 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.883615    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 07 11:47:25.883738 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.883634    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 07 11:47:25.883801 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.883778    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 07 11:47:25.883850 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.883825    2580 reconstruct.go:97] "Volume reconstruction finished"
Sep 07 11:47:25.883850 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.883832    2580 reconciler.go:29] "Reconciler: start to sync state"
Sep 07 11:47:25.884049 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.884033    2580 factory.go:55] Registering systemd factory
Sep 07 11:47:25.884049 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.884049    2580 factory.go:223] Registration of the systemd container factory successfully
Sep 07 11:47:25.884323 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.884308    2580 factory.go:153] Registering CRI-O factory
Sep 07 11:47:25.884417 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.884326    2580 factory.go:223] Registration of the crio container factory successfully
Sep 07 11:47:25.884417 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.884379    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 07 11:47:25.884417 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.884398    2580 factory.go:103] Registering Raw factory
Sep 07 11:47:25.884417 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.884415    2580 manager.go:1196] Started watching for new ooms in manager
Sep 07 11:47:25.884839 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.884825    2580 manager.go:349] Starting recovery of all containers
Sep 07 11:47:25.885383 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.885360    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gqgv5"
Sep 07 11:47:25.895033 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.895012    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gqgv5"
Sep 07 11:47:25.895149 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:25.895124    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-68.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 07 11:47:25.895272 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:25.895249    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 07 11:47:25.898633 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.898494    2580 manager.go:354] Recovery completed
Sep 07 11:47:25.903024 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.903011    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:47:25.905235 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.905213    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:47:25.905305 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.905251    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:47:25.905305 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.905267    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:47:25.905888 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.905873    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 07 11:47:25.905888 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.905887    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 07 11:47:25.906022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.905906    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 07 11:47:25.906022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.905886    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:47:25.906688 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.906654    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:47:25.906688 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.906677    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:47:25.906767 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.906690    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:47:25.907488 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:25.907405    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-68.ec2.internal.18d3076df8eb8827  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-68.ec2.internal,UID:ip-10-0-128-68.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-68.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-68.ec2.internal,},FirstTimestamp:2026-09-07 11:47:25.905233959 +0000 UTC m=+0.401123377,LastTimestamp:2026-09-07 11:47:25.905233959 +0000 UTC m=+0.401123377,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-68.ec2.internal,}"
Sep 07 11:47:25.907593 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.907581    2580 policy_none.go:49] "None policy: Start"
Sep 07 11:47:25.907632 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.907598    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 07 11:47:25.907632 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.907608    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 07 11:47:25.909025 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.909011    2580 policy_none.go:47] "Start"
Sep 07 11:47:25.937904 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.937886    2580 manager.go:337] "Starting Device Plugin manager"
Sep 07 11:47:25.951712 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:25.937921    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 07 11:47:25.951712 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.937932    2580 server.go:87] "Starting device plugin registration server"
Sep 07 11:47:25.951712 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.938238    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 07 11:47:25.951712 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.938251    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 07 11:47:25.951712 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.938840    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 07 11:47:25.951712 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.938934    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 07 11:47:25.951712 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:25.938944    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 07 11:47:25.951712 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:25.939011    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 07 11:47:25.951712 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:25.939062    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-68.ec2.internal\" not found"
Sep 07 11:47:26.002232 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.002184    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 07 11:47:26.003659 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.003632    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 07 11:47:26.003783 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.003694    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 07 11:47:26.003783 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.003726    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 07 11:47:26.003783 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:26.003770    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 07 11:47:26.005617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.005601    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:47:26.038573 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.038531    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:47:26.039529 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.039511    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:47:26.039612 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.039540    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:47:26.039612 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.039552    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:47:26.039612 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.039575    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.046852 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.046835    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.046943 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:26.046862    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-68.ec2.internal\": node \"ip-10-0-128-68.ec2.internal\" not found"
Sep 07 11:47:26.055835 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:26.055751    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-68.ec2.internal\" not found"
Sep 07 11:47:26.104295 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.104266    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-68.ec2.internal"]
Sep 07 11:47:26.104404 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.104333    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:47:26.105768 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.105750    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:47:26.105865 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.105778    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:47:26.105865 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.105788    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:47:26.107081 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.107067    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:47:26.107679 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.107663    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.107742 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.107702    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:47:26.107843 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.107802    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:47:26.107843 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.107836    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:47:26.107937 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.107851    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:47:26.108372 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.108358    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:47:26.108429 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.108385    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:47:26.108429 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.108395    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:47:26.109065 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.109049    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.109110 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.109079    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 11:47:26.109696 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.109678    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 11:47:26.109783 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.109711    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 11:47:26.109783 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.109729    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeHasSufficientPID"
Sep 07 11:47:26.129978 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:26.129949    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-68.ec2.internal\" not found" node="ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.134479 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:26.134460    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-68.ec2.internal\" not found" node="ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.156481 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:26.156454    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-68.ec2.internal\" not found"
Sep 07 11:47:26.186062 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.186036    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bc68d92aeaecfd0a0a6d199622a3c375-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal\" (UID: \"bc68d92aeaecfd0a0a6d199622a3c375\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.186135 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.186066    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc68d92aeaecfd0a0a6d199622a3c375-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal\" (UID: \"bc68d92aeaecfd0a0a6d199622a3c375\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.186135 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.186083    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/433f8b6f5fd9ac28e74f186701608de8-config\") pod \"kube-apiserver-proxy-ip-10-0-128-68.ec2.internal\" (UID: \"433f8b6f5fd9ac28e74f186701608de8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.257294 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:26.257259    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-68.ec2.internal\" not found"
Sep 07 11:47:26.286643 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.286605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bc68d92aeaecfd0a0a6d199622a3c375-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal\" (UID: \"bc68d92aeaecfd0a0a6d199622a3c375\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.286643 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.286649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc68d92aeaecfd0a0a6d199622a3c375-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal\" (UID: \"bc68d92aeaecfd0a0a6d199622a3c375\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.286765 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.286666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/433f8b6f5fd9ac28e74f186701608de8-config\") pod \"kube-apiserver-proxy-ip-10-0-128-68.ec2.internal\" (UID: \"433f8b6f5fd9ac28e74f186701608de8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.286765 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.286696    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/433f8b6f5fd9ac28e74f186701608de8-config\") pod \"kube-apiserver-proxy-ip-10-0-128-68.ec2.internal\" (UID: \"433f8b6f5fd9ac28e74f186701608de8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.286765 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.286712    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc68d92aeaecfd0a0a6d199622a3c375-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal\" (UID: \"bc68d92aeaecfd0a0a6d199622a3c375\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.286765 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.286712    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bc68d92aeaecfd0a0a6d199622a3c375-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal\" (UID: \"bc68d92aeaecfd0a0a6d199622a3c375\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.357968 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:26.357886    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-68.ec2.internal\" not found"
Sep 07 11:47:26.433333 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.433298    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.437695 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.437357    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-68.ec2.internal"
Sep 07 11:47:26.459278 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:26.459097    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-68.ec2.internal\" not found"
Sep 07 11:47:26.564694 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:26.564642    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-68.ec2.internal\" not found"
Sep 07 11:47:26.670175 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:26.670094    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-68.ec2.internal\" not found"
Sep 07 11:47:26.717440 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.717416    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:47:26.773229 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:26.773157    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-68.ec2.internal\" not found"
Sep 07 11:47:26.827062 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.827029    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 07 11:47:26.827557 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.827247    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 07 11:47:26.827557 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.827269    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 07 11:47:26.874196 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:26.874164    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-68.ec2.internal\" not found"
Sep 07 11:47:26.897776 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.897710    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-06 11:42:26 +0000 UTC" deadline="2028-04-15 13:56:14.809429002 +0000 UTC"
Sep 07 11:47:26.897776 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.897763    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14066h8m47.911671311s"
Sep 07 11:47:26.973197 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:26.973112    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:47:26.975127 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:26.975108    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-68.ec2.internal\" not found"
Sep 07 11:47:26.999091 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:26.999055    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod433f8b6f5fd9ac28e74f186701608de8.slice/crio-9d939da6f60eb63e577404d6647ccef206a9c257c5d83cb15d92ac6bf5335725 WatchSource:0}: Error finding container 9d939da6f60eb63e577404d6647ccef206a9c257c5d83cb15d92ac6bf5335725: Status 404 returned error can't find the container with id 9d939da6f60eb63e577404d6647ccef206a9c257c5d83cb15d92ac6bf5335725
Sep 07 11:47:27.004915 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.004893    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 11:47:27.007118 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.007079    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-68.ec2.internal" event={"ID":"433f8b6f5fd9ac28e74f186701608de8","Type":"ContainerStarted","Data":"9d939da6f60eb63e577404d6647ccef206a9c257c5d83cb15d92ac6bf5335725"}
Sep 07 11:47:27.076172 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:27.076135    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-68.ec2.internal\" not found"
Sep 07 11:47:27.090358 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.090330    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:47:27.183520 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.183491    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal"
Sep 07 11:47:27.195378 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.195351    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 07 11:47:27.196367 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.196353    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-68.ec2.internal"
Sep 07 11:47:27.203071 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.203053    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 07 11:47:27.265296 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:27.265264    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc68d92aeaecfd0a0a6d199622a3c375.slice/crio-eec6192049d942a38d6e66cda53408d364c5e834fae94085bf70cf44876e0991 WatchSource:0}: Error finding container eec6192049d942a38d6e66cda53408d364c5e834fae94085bf70cf44876e0991: Status 404 returned error can't find the container with id eec6192049d942a38d6e66cda53408d364c5e834fae94085bf70cf44876e0991
Sep 07 11:47:27.866106 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.866070    2580 apiserver.go:52] "Watching apiserver"
Sep 07 11:47:27.873965 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.873937    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 07 11:47:27.875230 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.875188    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-9f27p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal","openshift-multus/multus-additional-cni-plugins-4fj7g","openshift-network-diagnostics/network-check-target-q976p","kube-system/global-pull-secret-syncer-8s4jc","kube-system/konnectivity-agent-hwhqc","openshift-dns/node-resolver-kwr4n","openshift-multus/multus-mq4kn","openshift-multus/network-metrics-daemon-4dgj8","openshift-network-operator/iptables-alerter-nn82z","openshift-ovn-kubernetes/ovnkube-node-tjx47","kube-system/kube-apiserver-proxy-ip-10-0-128-68.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554","openshift-cluster-node-tuning-operator/tuned-dntlp"]
Sep 07 11:47:27.876722 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.876699    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kwr4n"
Sep 07 11:47:27.877953 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.877930    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:27.879839 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.879372    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lc7bd\""
Sep 07 11:47:27.879839 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.879409    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 07 11:47:27.879839 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.879444    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:27.879839 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.879579    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 07 11:47:27.880677 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.880492    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 07 11:47:27.880782 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.880695    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lwnfq\""
Sep 07 11:47:27.881082 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.880861    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 07 11:47:27.881082 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.880857    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 07 11:47:27.881270 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.881234    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:27.881347 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:27.881317    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:27.882042 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.882022    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 07 11:47:27.882133 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.882101    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 07 11:47:27.882344 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.882324    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hxmwm\""
Sep 07 11:47:27.882720 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.882440    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 07 11:47:27.882720 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.882533    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 07 11:47:27.883004 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.882870    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 07 11:47:27.883004 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.882941    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:27.883151 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:27.883018    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:27.883628 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.883607    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 07 11:47:27.884328 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.884308    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hwhqc"
Sep 07 11:47:27.885866 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.885724    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:27.885866 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:27.885819    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:47:27.886796 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.886688    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 07 11:47:27.886904 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.886847    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 07 11:47:27.886993 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.886960    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bp2z6\""
Sep 07 11:47:27.889889 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.887941    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9f27p"
Sep 07 11:47:27.890289 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.890261    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 07 11:47:27.890919 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.890828    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:27.891686 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.891658    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sq6dx\""
Sep 07 11:47:27.892980 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.892681    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 07 11:47:27.894512 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.893077    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nn82z"
Sep 07 11:47:27.894512 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.893665    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 07 11:47:27.894512 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.893931    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 07 11:47:27.894512 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.894447    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6dphq\""
Sep 07 11:47:27.896406 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.896274    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:27.897530 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.896783    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 07 11:47:27.897530 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.897344    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 07 11:47:27.897530 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.897430    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xmjvk\""
Sep 07 11:47:27.897853 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.897770    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 07 11:47:27.898166 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.898041    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 07 11:47:27.899730 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.899709    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 07 11:47:27.899974 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.899954    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 07 11:47:27.900122 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.900100    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 07 11:47:27.900316 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.900297    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xc82c\""
Sep 07 11:47:27.900704 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.900681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a8041043-f46e-4a5f-8e1f-01018a5189bb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:27.900802 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.900735    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:27.900802 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.900778    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hslsx\" (UniqueName: \"kubernetes.io/projected/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-kube-api-access-hslsx\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:27.900904 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.900806    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8041043-f46e-4a5f-8e1f-01018a5189bb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:27.900904 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.900842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5p9j\" (UniqueName: \"kubernetes.io/projected/a8041043-f46e-4a5f-8e1f-01018a5189bb-kube-api-access-h5p9j\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:27.900904 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.900867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkzs7\" (UniqueName: \"kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7\") pod \"network-check-target-q976p\" (UID: \"96cc191d-da2d-4e0e-9995-bde02f99e254\") " pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:27.900904 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.900891    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ec37e993-169b-4dc5-b5f6-e938f46e33d7-dbus\") pod \"global-pull-secret-syncer-8s4jc\" (UID: \"ec37e993-169b-4dc5-b5f6-e938f46e33d7\") " pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:27.901027 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.900914    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c2e42f2-bf8f-4093-b496-af2041e0090e-tmp-dir\") pod \"node-resolver-kwr4n\" (UID: \"3c2e42f2-bf8f-4093-b496-af2041e0090e\") " pod="openshift-dns/node-resolver-kwr4n"
Sep 07 11:47:27.901027 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.900936    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-etc-selinux\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:27.901027 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.900957    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a8041043-f46e-4a5f-8e1f-01018a5189bb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:27.901027 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.900981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret\") pod \"global-pull-secret-syncer-8s4jc\" (UID: \"ec37e993-169b-4dc5-b5f6-e938f46e33d7\") " pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:27.901162 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.901037    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8a42548a-1c35-4dc8-8ddd-8d6777ac544e-konnectivity-ca\") pod \"konnectivity-agent-hwhqc\" (UID: \"8a42548a-1c35-4dc8-8ddd-8d6777ac544e\") " pod="kube-system/konnectivity-agent-hwhqc"
Sep 07 11:47:27.901162 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.901074    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3c2e42f2-bf8f-4093-b496-af2041e0090e-hosts-file\") pod \"node-resolver-kwr4n\" (UID: \"3c2e42f2-bf8f-4093-b496-af2041e0090e\") " pod="openshift-dns/node-resolver-kwr4n"
Sep 07 11:47:27.901162 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.901099    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r5g6\" (UniqueName: \"kubernetes.io/projected/3c2e42f2-bf8f-4093-b496-af2041e0090e-kube-api-access-4r5g6\") pod \"node-resolver-kwr4n\" (UID: \"3c2e42f2-bf8f-4093-b496-af2041e0090e\") " pod="openshift-dns/node-resolver-kwr4n"
Sep 07 11:47:27.901162 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.901151    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8041043-f46e-4a5f-8e1f-01018a5189bb-system-cni-dir\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:27.901333 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.901201    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8041043-f46e-4a5f-8e1f-01018a5189bb-cni-binary-copy\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:27.901333 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.901238    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ec37e993-169b-4dc5-b5f6-e938f46e33d7-kubelet-config\") pod \"global-pull-secret-syncer-8s4jc\" (UID: \"ec37e993-169b-4dc5-b5f6-e938f46e33d7\") " pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:27.901333 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.901268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8a42548a-1c35-4dc8-8ddd-8d6777ac544e-agent-certs\") pod \"konnectivity-agent-hwhqc\" (UID: \"8a42548a-1c35-4dc8-8ddd-8d6777ac544e\") " pod="kube-system/konnectivity-agent-hwhqc"
Sep 07 11:47:27.901507 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.901341    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-socket-dir\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:27.901507 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.901366    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-registration-dir\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:27.901507 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.901393    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-device-dir\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:27.901507 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.901420    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-sys-fs\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:27.901701 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.901508    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8041043-f46e-4a5f-8e1f-01018a5189bb-cnibin\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:27.901701 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.901581    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 07 11:47:27.902820 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.902120    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 07 11:47:27.902820 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.902142    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 07 11:47:27.902820 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.902313    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:27.902820 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.902767    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 07 11:47:27.904878 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.904858    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 07 11:47:27.905127 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.905109    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-trqvc\""
Sep 07 11:47:27.985303 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:27.985272    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 07 11:47:28.002371 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.002338    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg42t\" (UniqueName: \"kubernetes.io/projected/5250f748-f185-42d5-bcc6-006887052b3c-kube-api-access-rg42t\") pod \"iptables-alerter-nn82z\" (UID: \"5250f748-f185-42d5-bcc6-006887052b3c\") " pod="openshift-network-operator/iptables-alerter-nn82z"
Sep 07 11:47:28.002534 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.002380    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s77sl\" (UniqueName: \"kubernetes.io/projected/fff6de68-6332-4f28-a0cb-60625f82ac14-kube-api-access-s77sl\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.002534 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.002411    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-run-openvswitch\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.002534 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.002472    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-hostroot\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.002716 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.002528    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5250f748-f185-42d5-bcc6-006887052b3c-iptables-alerter-script\") pod \"iptables-alerter-nn82z\" (UID: \"5250f748-f185-42d5-bcc6-006887052b3c\") " pod="openshift-network-operator/iptables-alerter-nn82z"
Sep 07 11:47:28.002716 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.002568    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fff6de68-6332-4f28-a0cb-60625f82ac14-tmp\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.002716 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.002606    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8041043-f46e-4a5f-8e1f-01018a5189bb-cni-binary-copy\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:28.002716 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.002665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-cnibin\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.002716 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.002690    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-cni-bin\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.002716 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.002716    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-device-dir\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:28.002982 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.002829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5p9j\" (UniqueName: \"kubernetes.io/projected/a8041043-f46e-4a5f-8e1f-01018a5189bb-kube-api-access-h5p9j\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:28.002982 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.002873    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs\") pod \"network-metrics-daemon-4dgj8\" (UID: \"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f\") " pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:28.002982 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.002907    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-device-dir\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:28.002982 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.002900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:28.002982 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.002975    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:28.003359 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.003328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-multus-socket-dir-parent\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.003469 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.003370    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-run-netns\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.003469 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.003429    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-systemd-units\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.003469 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.003455    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec0a84de-962e-47ab-9639-fc0d09e06b5d-ovnkube-config\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.003622 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.003518    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec0a84de-962e-47ab-9639-fc0d09e06b5d-ovnkube-script-lib\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.003622 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.003580    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkzs7\" (UniqueName: \"kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7\") pod \"network-check-target-q976p\" (UID: \"96cc191d-da2d-4e0e-9995-bde02f99e254\") " pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:28.003731 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.003649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-run-k8s-cni-cncf-io\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.003822 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.003796    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-multus-daemon-config\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.003953 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.003900    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8722b24c-1b27-4443-aacb-b96b30a04bf3-serviceca\") pod \"node-ca-9f27p\" (UID: \"8722b24c-1b27-4443-aacb-b96b30a04bf3\") " pod="openshift-image-registry/node-ca-9f27p"
Sep 07 11:47:28.003953 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.003945    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret\") pod \"global-pull-secret-syncer-8s4jc\" (UID: \"ec37e993-169b-4dc5-b5f6-e938f46e33d7\") " pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:28.004101 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.003979    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-sysconfig\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.004101 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004003    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-lib-modules\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.004101 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004032    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-run-ovn-kubernetes\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.004275 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004103    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wctqj\" (UniqueName: \"kubernetes.io/projected/ec0a84de-962e-47ab-9639-fc0d09e06b5d-kube-api-access-wctqj\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.004275 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004188    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ec37e993-169b-4dc5-b5f6-e938f46e33d7-kubelet-config\") pod \"global-pull-secret-syncer-8s4jc\" (UID: \"ec37e993-169b-4dc5-b5f6-e938f46e33d7\") " pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:28.004275 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004241    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8041043-f46e-4a5f-8e1f-01018a5189bb-cni-binary-copy\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:28.004430 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:28.004302    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:28.004430 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004347    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjbtk\" (UniqueName: \"kubernetes.io/projected/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-kube-api-access-hjbtk\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.004430 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004395    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ec37e993-169b-4dc5-b5f6-e938f46e33d7-kubelet-config\") pod \"global-pull-secret-syncer-8s4jc\" (UID: \"ec37e993-169b-4dc5-b5f6-e938f46e33d7\") " pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:28.004430 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:28.004394    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret podName:ec37e993-169b-4dc5-b5f6-e938f46e33d7 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:28.504355063 +0000 UTC m=+3.000244478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret") pod "global-pull-secret-syncer-8s4jc" (UID: "ec37e993-169b-4dc5-b5f6-e938f46e33d7") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:28.004641 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004439    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-kubernetes\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.004641 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004499    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-kubelet\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.004641 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004537    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-run-systemd\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.004641 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004561    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-etc-openvswitch\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.004641 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004604    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-cni-netd\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.004937 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004639    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-sysctl-d\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.004937 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004689    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-node-log\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.004937 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-log-socket\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.004937 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004761    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-multus-cni-dir\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.004937 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004791    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-var-lib-cni-multus\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.004937 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8a42548a-1c35-4dc8-8ddd-8d6777ac544e-konnectivity-ca\") pod \"konnectivity-agent-hwhqc\" (UID: \"8a42548a-1c35-4dc8-8ddd-8d6777ac544e\") " pod="kube-system/konnectivity-agent-hwhqc"
Sep 07 11:47:28.005161 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.004934    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8041043-f46e-4a5f-8e1f-01018a5189bb-system-cni-dir\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:28.005161 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005000    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8041043-f46e-4a5f-8e1f-01018a5189bb-system-cni-dir\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:28.005161 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005033    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5250f748-f185-42d5-bcc6-006887052b3c-host-slash\") pod \"iptables-alerter-nn82z\" (UID: \"5250f748-f185-42d5-bcc6-006887052b3c\") " pod="openshift-network-operator/iptables-alerter-nn82z"
Sep 07 11:47:28.005161 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005086    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-slash\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.005161 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8a42548a-1c35-4dc8-8ddd-8d6777ac544e-agent-certs\") pod \"konnectivity-agent-hwhqc\" (UID: \"8a42548a-1c35-4dc8-8ddd-8d6777ac544e\") " pod="kube-system/konnectivity-agent-hwhqc"
Sep 07 11:47:28.005345 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-socket-dir\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:28.005345 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005285    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-var-lib-kubelet\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.005345 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005308    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-run-multus-certs\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.005473 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005370    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-etc-kubernetes\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.005473 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005397    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dgnl\" (UniqueName: \"kubernetes.io/projected/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-kube-api-access-6dgnl\") pod \"network-metrics-daemon-4dgj8\" (UID: \"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f\") " pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:28.005473 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-modprobe-d\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.005621 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005463    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-systemd\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.005621 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005527    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-var-lib-cni-bin\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.005621 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005521    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8a42548a-1c35-4dc8-8ddd-8d6777ac544e-konnectivity-ca\") pod \"konnectivity-agent-hwhqc\" (UID: \"8a42548a-1c35-4dc8-8ddd-8d6777ac544e\") " pod="kube-system/konnectivity-agent-hwhqc"
Sep 07 11:47:28.005621 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005560    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smgq8\" (UniqueName: \"kubernetes.io/projected/8722b24c-1b27-4443-aacb-b96b30a04bf3-kube-api-access-smgq8\") pod \"node-ca-9f27p\" (UID: \"8722b24c-1b27-4443-aacb-b96b30a04bf3\") " pod="openshift-image-registry/node-ca-9f27p"
Sep 07 11:47:28.005821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-sys\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.005821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-tuned\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.005821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005693    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-var-lib-openvswitch\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.005821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005716    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-socket-dir\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:28.005821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005743    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.006045 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005826    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ec37e993-169b-4dc5-b5f6-e938f46e33d7-dbus\") pod \"global-pull-secret-syncer-8s4jc\" (UID: \"ec37e993-169b-4dc5-b5f6-e938f46e33d7\") " pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:28.006045 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005910    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c2e42f2-bf8f-4093-b496-af2041e0090e-tmp-dir\") pod \"node-resolver-kwr4n\" (UID: \"3c2e42f2-bf8f-4093-b496-af2041e0090e\") " pod="openshift-dns/node-resolver-kwr4n"
Sep 07 11:47:28.006045 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005939    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-etc-selinux\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:28.006045 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006000    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 07 11:47:28.006045 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.005990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a8041043-f46e-4a5f-8e1f-01018a5189bb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:28.006301 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-host\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.006301 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006103    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec0a84de-962e-47ab-9639-fc0d09e06b5d-ovn-node-metrics-cert\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.006301 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3c2e42f2-bf8f-4093-b496-af2041e0090e-hosts-file\") pod \"node-resolver-kwr4n\" (UID: \"3c2e42f2-bf8f-4093-b496-af2041e0090e\") " pod="openshift-dns/node-resolver-kwr4n"
Sep 07 11:47:28.006301 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r5g6\" (UniqueName: \"kubernetes.io/projected/3c2e42f2-bf8f-4093-b496-af2041e0090e-kube-api-access-4r5g6\") pod \"node-resolver-kwr4n\" (UID: \"3c2e42f2-bf8f-4093-b496-af2041e0090e\") " pod="openshift-dns/node-resolver-kwr4n"
Sep 07 11:47:28.006511 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-system-cni-dir\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.006511 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006356    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-cni-binary-copy\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.006511 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-sysctl-conf\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.006511 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-run\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.006511 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006438    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec0a84de-962e-47ab-9639-fc0d09e06b5d-env-overrides\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.006757 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006459    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c2e42f2-bf8f-4093-b496-af2041e0090e-tmp-dir\") pod \"node-resolver-kwr4n\" (UID: \"3c2e42f2-bf8f-4093-b496-af2041e0090e\") " pod="openshift-dns/node-resolver-kwr4n"
Sep 07 11:47:28.006757 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-registration-dir\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:28.006757 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-sys-fs\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:28.006757 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006555    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3c2e42f2-bf8f-4093-b496-af2041e0090e-hosts-file\") pod \"node-resolver-kwr4n\" (UID: \"3c2e42f2-bf8f-4093-b496-af2041e0090e\") " pod="openshift-dns/node-resolver-kwr4n"
Sep 07 11:47:28.006757 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-sys-fs\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:28.006757 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8041043-f46e-4a5f-8e1f-01018a5189bb-cnibin\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:28.006757 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006699    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8041043-f46e-4a5f-8e1f-01018a5189bb-cnibin\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:28.006757 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006717    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a8041043-f46e-4a5f-8e1f-01018a5189bb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:28.006757 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006151    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ec37e993-169b-4dc5-b5f6-e938f46e33d7-dbus\") pod \"global-pull-secret-syncer-8s4jc\" (UID: \"ec37e993-169b-4dc5-b5f6-e938f46e33d7\") " pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:28.007235 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006904    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-var-lib-kubelet\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.007235 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006929    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-registration-dir\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:28.007235 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006950    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-run-netns\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.007235 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006962    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-etc-selinux\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:28.007235 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.006977    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-run-ovn\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.007235 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.007116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hslsx\" (UniqueName: \"kubernetes.io/projected/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-kube-api-access-hslsx\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:28.007505 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.007322    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8041043-f46e-4a5f-8e1f-01018a5189bb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:28.007505 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.007390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-multus-conf-dir\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.007505 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.007451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8722b24c-1b27-4443-aacb-b96b30a04bf3-host\") pod \"node-ca-9f27p\" (UID: \"8722b24c-1b27-4443-aacb-b96b30a04bf3\") " pod="openshift-image-registry/node-ca-9f27p"
Sep 07 11:47:28.007646 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.007628    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a8041043-f46e-4a5f-8e1f-01018a5189bb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:28.007717 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.007686    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a8041043-f46e-4a5f-8e1f-01018a5189bb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:28.007717 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.007695    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8041043-f46e-4a5f-8e1f-01018a5189bb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:28.011851 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.011818    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal" event={"ID":"bc68d92aeaecfd0a0a6d199622a3c375","Type":"ContainerStarted","Data":"eec6192049d942a38d6e66cda53408d364c5e834fae94085bf70cf44876e0991"}
Sep 07 11:47:28.012039 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.012017    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8a42548a-1c35-4dc8-8ddd-8d6777ac544e-agent-certs\") pod \"konnectivity-agent-hwhqc\" (UID: \"8a42548a-1c35-4dc8-8ddd-8d6777ac544e\") " pod="kube-system/konnectivity-agent-hwhqc"
Sep 07 11:47:28.015761 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.015729    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5p9j\" (UniqueName: \"kubernetes.io/projected/a8041043-f46e-4a5f-8e1f-01018a5189bb-kube-api-access-h5p9j\") pod \"multus-additional-cni-plugins-4fj7g\" (UID: \"a8041043-f46e-4a5f-8e1f-01018a5189bb\") " pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:28.020272 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:28.020249    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:47:28.020400 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:28.020279    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:47:28.020400 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:28.020312    2580 projected.go:196] Error preparing data for projected volume kube-api-access-dkzs7 for pod openshift-network-diagnostics/network-check-target-q976p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:28.020540 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:28.020519    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7 podName:96cc191d-da2d-4e0e-9995-bde02f99e254 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:28.520497902 +0000 UTC m=+3.016387329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dkzs7" (UniqueName: "kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7") pod "network-check-target-q976p" (UID: "96cc191d-da2d-4e0e-9995-bde02f99e254") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:28.024214 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.024194    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hslsx\" (UniqueName: \"kubernetes.io/projected/c7a26d7e-e575-483f-ae74-ab2a1959e8d9-kube-api-access-hslsx\") pod \"aws-ebs-csi-driver-node-jg554\" (UID: \"c7a26d7e-e575-483f-ae74-ab2a1959e8d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:28.025151 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.025128    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r5g6\" (UniqueName: \"kubernetes.io/projected/3c2e42f2-bf8f-4093-b496-af2041e0090e-kube-api-access-4r5g6\") pod \"node-resolver-kwr4n\" (UID: \"3c2e42f2-bf8f-4093-b496-af2041e0090e\") " pod="openshift-dns/node-resolver-kwr4n"
Sep 07 11:47:28.031789 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.031761    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cmmxr"
Sep 07 11:47:28.039306 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.039257    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cmmxr"
Sep 07 11:47:28.107931 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.107889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjbtk\" (UniqueName: \"kubernetes.io/projected/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-kube-api-access-hjbtk\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.108127 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.107943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-kubernetes\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.108127 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.107980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-kubelet\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.108127 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108002    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-run-systemd\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.108127 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108022    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-etc-openvswitch\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.108127 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-cni-netd\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.108127 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108064    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-sysctl-d\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.108127 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108101    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-node-log\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.108127 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108104    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-kubernetes\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.108127 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108123    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-log-socket\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108146    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-multus-cni-dir\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-var-lib-cni-multus\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108281    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5250f748-f185-42d5-bcc6-006887052b3c-host-slash\") pod \"iptables-alerter-nn82z\" (UID: \"5250f748-f185-42d5-bcc6-006887052b3c\") " pod="openshift-network-operator/iptables-alerter-nn82z"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108282    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-cni-netd\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108308    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-slash\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-var-lib-kubelet\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108348    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-kubelet\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-run-multus-certs\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-etc-kubernetes\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108426    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-multus-cni-dir\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-etc-openvswitch\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dgnl\" (UniqueName: \"kubernetes.io/projected/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-kube-api-access-6dgnl\") pod \"network-metrics-daemon-4dgj8\" (UID: \"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f\") " pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-node-log\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-modprobe-d\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108542    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-systemd\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108600    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-systemd\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.108617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108615    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-modprobe-d\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-var-lib-cni-bin\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108667    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smgq8\" (UniqueName: \"kubernetes.io/projected/8722b24c-1b27-4443-aacb-b96b30a04bf3-kube-api-access-smgq8\") pod \"node-ca-9f27p\" (UID: \"8722b24c-1b27-4443-aacb-b96b30a04bf3\") " pod="openshift-image-registry/node-ca-9f27p"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108672    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-sysctl-d\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108675    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-var-lib-kubelet\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108696    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-sys\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108719    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5250f748-f185-42d5-bcc6-006887052b3c-host-slash\") pod \"iptables-alerter-nn82z\" (UID: \"5250f748-f185-42d5-bcc6-006887052b3c\") " pod="openshift-network-operator/iptables-alerter-nn82z"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108720    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-log-socket\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108737    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-etc-kubernetes\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108404    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-run-systemd\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108751    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-var-lib-cni-multus\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-tuned\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108801    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-sys\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108807    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-run-multus-certs\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-var-lib-openvswitch\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108831    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-var-lib-cni-bin\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108844    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-host\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.109286 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108882    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108847    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-var-lib-openvswitch\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108899    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec0a84de-962e-47ab-9639-fc0d09e06b5d-ovn-node-metrics-cert\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108927    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-system-cni-dir\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108954    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-cni-binary-copy\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-sysctl-conf\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.108930    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-host\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109003    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-run\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec0a84de-962e-47ab-9639-fc0d09e06b5d-env-overrides\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109044    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-system-cni-dir\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109056    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-var-lib-kubelet\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-run-netns\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-run-ovn\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109128    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-multus-conf-dir\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109150    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-sysctl-conf\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8722b24c-1b27-4443-aacb-b96b30a04bf3-host\") pod \"node-ca-9f27p\" (UID: \"8722b24c-1b27-4443-aacb-b96b30a04bf3\") " pod="openshift-image-registry/node-ca-9f27p"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109178    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg42t\" (UniqueName: \"kubernetes.io/projected/5250f748-f185-42d5-bcc6-006887052b3c-kube-api-access-rg42t\") pod \"iptables-alerter-nn82z\" (UID: \"5250f748-f185-42d5-bcc6-006887052b3c\") " pod="openshift-network-operator/iptables-alerter-nn82z"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109189    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8722b24c-1b27-4443-aacb-b96b30a04bf3-host\") pod \"node-ca-9f27p\" (UID: \"8722b24c-1b27-4443-aacb-b96b30a04bf3\") " pod="openshift-image-registry/node-ca-9f27p"
Sep 07 11:47:28.110198 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109201    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77sl\" (UniqueName: \"kubernetes.io/projected/fff6de68-6332-4f28-a0cb-60625f82ac14-kube-api-access-s77sl\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-run-openvswitch\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-hostroot\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109325    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-hostroot\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109335    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-run-netns\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5250f748-f185-42d5-bcc6-006887052b3c-iptables-alerter-script\") pod \"iptables-alerter-nn82z\" (UID: \"5250f748-f185-42d5-bcc6-006887052b3c\") " pod="openshift-network-operator/iptables-alerter-nn82z"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109431    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fff6de68-6332-4f28-a0cb-60625f82ac14-tmp\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-cnibin\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-cni-bin\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs\") pod \"network-metrics-daemon-4dgj8\" (UID: \"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f\") " pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-multus-socket-dir-parent\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109594    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-run-netns\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-systemd-units\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec0a84de-962e-47ab-9639-fc0d09e06b5d-ovnkube-config\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec0a84de-962e-47ab-9639-fc0d09e06b5d-ovnkube-script-lib\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109708    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec0a84de-962e-47ab-9639-fc0d09e06b5d-env-overrides\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-run-k8s-cni-cncf-io\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109740    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-multus-daemon-config\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.111156 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8722b24c-1b27-4443-aacb-b96b30a04bf3-serviceca\") pod \"node-ca-9f27p\" (UID: \"8722b24c-1b27-4443-aacb-b96b30a04bf3\") " pod="openshift-image-registry/node-ca-9f27p"
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109754    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-run-ovn\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-multus-conf-dir\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109813    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-sysconfig\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109839    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-lib-modules\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109866    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-run-ovn-kubernetes\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109918    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wctqj\" (UniqueName: \"kubernetes.io/projected/ec0a84de-962e-47ab-9639-fc0d09e06b5d-kube-api-access-wctqj\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.109924    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-cni-binary-copy\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.110012    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-var-lib-kubelet\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.110431    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec0a84de-962e-47ab-9639-fc0d09e06b5d-ovnkube-config\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.110502    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-run-ovn-kubernetes\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.110529    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5250f748-f185-42d5-bcc6-006887052b3c-iptables-alerter-script\") pod \"iptables-alerter-nn82z\" (UID: \"5250f748-f185-42d5-bcc6-006887052b3c\") " pod="openshift-network-operator/iptables-alerter-nn82z"
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.110561    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-multus-socket-dir-parent\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.110605    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-cnibin\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.110638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-cni-bin\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:28.110693    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:28.110743    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs podName:7e4e91b4-23e4-4903-8e2d-cf1d2145b51f nodeName:}" failed. No retries permitted until 2026-09-07 11:47:28.610730596 +0000 UTC m=+3.106620000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs") pod "network-metrics-daemon-4dgj8" (UID: "7e4e91b4-23e4-4903-8e2d-cf1d2145b51f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:28.112022 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.110883    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-systemd-units\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.112859 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.110910    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-run-netns\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.112859 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.110945    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-host-run-k8s-cni-cncf-io\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.112859 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.111046    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec0a84de-962e-47ab-9639-fc0d09e06b5d-ovnkube-script-lib\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.112859 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.111175    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-lib-modules\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.112859 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.111217    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8722b24c-1b27-4443-aacb-b96b30a04bf3-serviceca\") pod \"node-ca-9f27p\" (UID: \"8722b24c-1b27-4443-aacb-b96b30a04bf3\") " pod="openshift-image-registry/node-ca-9f27p"
Sep 07 11:47:28.112859 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.111241    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-sysconfig\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.112859 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.111256    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fff6de68-6332-4f28-a0cb-60625f82ac14-run\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.112859 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.111300    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-run-openvswitch\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.112859 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.111329    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-multus-daemon-config\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.112859 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.111348    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec0a84de-962e-47ab-9639-fc0d09e06b5d-host-slash\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.112859 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.111969    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fff6de68-6332-4f28-a0cb-60625f82ac14-etc-tuned\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.113432 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.113141    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fff6de68-6332-4f28-a0cb-60625f82ac14-tmp\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.114105 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.114066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec0a84de-962e-47ab-9639-fc0d09e06b5d-ovn-node-metrics-cert\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.123641 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.123563    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dgnl\" (UniqueName: \"kubernetes.io/projected/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-kube-api-access-6dgnl\") pod \"network-metrics-daemon-4dgj8\" (UID: \"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f\") " pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:28.124315 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.124273    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjbtk\" (UniqueName: \"kubernetes.io/projected/7300402b-f9fd-4f08-8728-b3a4ad8e2d8e-kube-api-access-hjbtk\") pod \"multus-mq4kn\" (UID: \"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e\") " pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.124645 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.124622    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smgq8\" (UniqueName: \"kubernetes.io/projected/8722b24c-1b27-4443-aacb-b96b30a04bf3-kube-api-access-smgq8\") pod \"node-ca-9f27p\" (UID: \"8722b24c-1b27-4443-aacb-b96b30a04bf3\") " pod="openshift-image-registry/node-ca-9f27p"
Sep 07 11:47:28.125733 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.125708    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg42t\" (UniqueName: \"kubernetes.io/projected/5250f748-f185-42d5-bcc6-006887052b3c-kube-api-access-rg42t\") pod \"iptables-alerter-nn82z\" (UID: \"5250f748-f185-42d5-bcc6-006887052b3c\") " pod="openshift-network-operator/iptables-alerter-nn82z"
Sep 07 11:47:28.126677 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.126654    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s77sl\" (UniqueName: \"kubernetes.io/projected/fff6de68-6332-4f28-a0cb-60625f82ac14-kube-api-access-s77sl\") pod \"tuned-dntlp\" (UID: \"fff6de68-6332-4f28-a0cb-60625f82ac14\") " pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.126777 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.126713    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wctqj\" (UniqueName: \"kubernetes.io/projected/ec0a84de-962e-47ab-9639-fc0d09e06b5d-kube-api-access-wctqj\") pod \"ovnkube-node-tjx47\" (UID: \"ec0a84de-962e-47ab-9639-fc0d09e06b5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.192883 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.192845    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kwr4n"
Sep 07 11:47:28.201956 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.201931    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554"
Sep 07 11:47:28.211362 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.211331    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4fj7g"
Sep 07 11:47:28.218546 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.218520    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hwhqc"
Sep 07 11:47:28.227380 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.227353    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9f27p"
Sep 07 11:47:28.235362 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.235331    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dntlp"
Sep 07 11:47:28.245284 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.245259    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nn82z"
Sep 07 11:47:28.254250 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.254231    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:28.260294 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.260274    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mq4kn"
Sep 07 11:47:28.297739 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.297706    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:47:28.514491 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.514403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret\") pod \"global-pull-secret-syncer-8s4jc\" (UID: \"ec37e993-169b-4dc5-b5f6-e938f46e33d7\") " pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:28.514655 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:28.514542    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:28.514655 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:28.514606    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret podName:ec37e993-169b-4dc5-b5f6-e938f46e33d7 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:29.514588545 +0000 UTC m=+4.010477947 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret") pod "global-pull-secret-syncer-8s4jc" (UID: "ec37e993-169b-4dc5-b5f6-e938f46e33d7") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:28.614715 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.614682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs\") pod \"network-metrics-daemon-4dgj8\" (UID: \"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f\") " pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:28.614885 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:28.614724    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkzs7\" (UniqueName: \"kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7\") pod \"network-check-target-q976p\" (UID: \"96cc191d-da2d-4e0e-9995-bde02f99e254\") " pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:28.614885 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:28.614864    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:28.614885 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:28.614874    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:47:28.615003 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:28.614892    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:47:28.615003 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:28.614906    2580 projected.go:196] Error preparing data for projected volume kube-api-access-dkzs7 for pod openshift-network-diagnostics/network-check-target-q976p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:28.615003 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:28.614929    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs podName:7e4e91b4-23e4-4903-8e2d-cf1d2145b51f nodeName:}" failed. No retries permitted until 2026-09-07 11:47:29.614911786 +0000 UTC m=+4.110801207 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs") pod "network-metrics-daemon-4dgj8" (UID: "7e4e91b4-23e4-4903-8e2d-cf1d2145b51f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:28.615003 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:28.614948    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7 podName:96cc191d-da2d-4e0e-9995-bde02f99e254 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:29.61493793 +0000 UTC m=+4.110827334 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dkzs7" (UniqueName: "kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7") pod "network-check-target-q976p" (UID: "96cc191d-da2d-4e0e-9995-bde02f99e254") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:29.002354 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:29.002297    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfff6de68_6332_4f28_a0cb_60625f82ac14.slice/crio-985767aa3a1ce3e831a5109b87f9e5fc01f82a777f80cefa40e4450206ef1eb3 WatchSource:0}: Error finding container 985767aa3a1ce3e831a5109b87f9e5fc01f82a777f80cefa40e4450206ef1eb3: Status 404 returned error can't find the container with id 985767aa3a1ce3e831a5109b87f9e5fc01f82a777f80cefa40e4450206ef1eb3
Sep 07 11:47:29.003995 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:29.003907    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:29.004053 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:29.004034    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:29.005519 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:47:29.005384    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7a26d7e_e575_483f_ae74_ab2a1959e8d9.slice/crio-d8708fc404d7a464e6e178e48e526f053c8cc1701960114a81746fa6f8537416 WatchSource:0}: Error finding container d8708fc404d7a464e6e178e48e526f053c8cc1701960114a81746fa6f8537416: Status 404 returned error can't find the container with id d8708fc404d7a464e6e178e48e526f053c8cc1701960114a81746fa6f8537416
Sep 07 11:47:29.020731 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:29.020702    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dntlp" event={"ID":"fff6de68-6332-4f28-a0cb-60625f82ac14","Type":"ContainerStarted","Data":"985767aa3a1ce3e831a5109b87f9e5fc01f82a777f80cefa40e4450206ef1eb3"}
Sep 07 11:47:29.024588 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:29.024525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hwhqc" event={"ID":"8a42548a-1c35-4dc8-8ddd-8d6777ac544e","Type":"ContainerStarted","Data":"c8e6664c3881a0a0ba72dceba4117231a111e91fa3fe13d37e1376fb1417ee35"}
Sep 07 11:47:29.026173 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:29.026143    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9f27p" event={"ID":"8722b24c-1b27-4443-aacb-b96b30a04bf3","Type":"ContainerStarted","Data":"b8640fb482a890717abd8084aa6903316bab2c6e32cdda2706fd7ab787930820"}
Sep 07 11:47:29.029258 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:29.029185    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mq4kn" event={"ID":"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e","Type":"ContainerStarted","Data":"f4915af10b52ebe8e723e72a07b9f0d9504520912671b090d37fbb1ea3814b9a"}
Sep 07 11:47:29.031161 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:29.031125    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554" event={"ID":"c7a26d7e-e575-483f-ae74-ab2a1959e8d9","Type":"ContainerStarted","Data":"d8708fc404d7a464e6e178e48e526f053c8cc1701960114a81746fa6f8537416"}
Sep 07 11:47:29.042796 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:29.042765    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 11:42:28 +0000 UTC" deadline="2028-06-05 08:56:46.929770396 +0000 UTC"
Sep 07 11:47:29.042880 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:29.042797    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15285h9m17.886977023s"
Sep 07 11:47:29.113944 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:29.113916    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 11:47:29.523023 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:29.522721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret\") pod \"global-pull-secret-syncer-8s4jc\" (UID: \"ec37e993-169b-4dc5-b5f6-e938f46e33d7\") " pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:29.523182 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:29.522940    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:29.523182 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:29.523092    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret podName:ec37e993-169b-4dc5-b5f6-e938f46e33d7 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:31.523071873 +0000 UTC m=+6.018961277 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret") pod "global-pull-secret-syncer-8s4jc" (UID: "ec37e993-169b-4dc5-b5f6-e938f46e33d7") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:29.624249 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:29.624211    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs\") pod \"network-metrics-daemon-4dgj8\" (UID: \"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f\") " pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:29.624422 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:29.624261    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkzs7\" (UniqueName: \"kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7\") pod \"network-check-target-q976p\" (UID: \"96cc191d-da2d-4e0e-9995-bde02f99e254\") " pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:29.624510 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:29.624442    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:47:29.624510 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:29.624461    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:47:29.624510 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:29.624474    2580 projected.go:196] Error preparing data for projected volume kube-api-access-dkzs7 for pod openshift-network-diagnostics/network-check-target-q976p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:29.624667 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:29.624530    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7 podName:96cc191d-da2d-4e0e-9995-bde02f99e254 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:31.624512514 +0000 UTC m=+6.120401921 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dkzs7" (UniqueName: "kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7") pod "network-check-target-q976p" (UID: "96cc191d-da2d-4e0e-9995-bde02f99e254") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:29.624934 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:29.624914    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:29.624995 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:29.624971    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs podName:7e4e91b4-23e4-4903-8e2d-cf1d2145b51f nodeName:}" failed. No retries permitted until 2026-09-07 11:47:31.62495616 +0000 UTC m=+6.120845566 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs") pod "network-metrics-daemon-4dgj8" (UID: "7e4e91b4-23e4-4903-8e2d-cf1d2145b51f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:30.007213 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:30.006339    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:30.007213 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:30.006470    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:30.007213 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:30.006895    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:30.007213 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:30.007021    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:47:30.038072 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:30.038032    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-68.ec2.internal" event={"ID":"433f8b6f5fd9ac28e74f186701608de8","Type":"ContainerStarted","Data":"d22791b42e7a8c5a7a425590528a00f98c944a0ffa38c426388df9cb028d294d"}
Sep 07 11:47:30.043307 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:30.043264    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 11:42:28 +0000 UTC" deadline="2028-03-07 03:08:34.82897511 +0000 UTC"
Sep 07 11:47:30.043307 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:30.043305    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13119h21m4.785674284s"
Sep 07 11:47:30.053895 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:30.053861    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4fj7g" event={"ID":"a8041043-f46e-4a5f-8e1f-01018a5189bb","Type":"ContainerStarted","Data":"bf817903457a4499be49d99e7e5a0657da46828310c6e37b505559efbbf8edcd"}
Sep 07 11:47:30.066575 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:30.066538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47" event={"ID":"ec0a84de-962e-47ab-9639-fc0d09e06b5d","Type":"ContainerStarted","Data":"fe414b9d4125c4797463f326a158f0f028fae7ae7169d6f2d362f1b134db6ffd"}
Sep 07 11:47:30.083564 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:30.083528    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kwr4n" event={"ID":"3c2e42f2-bf8f-4093-b496-af2041e0090e","Type":"ContainerStarted","Data":"71f57aa25c9c8b344ce5d8648095a3f9e07f9529fd075ba95ac0095a52183a42"}
Sep 07 11:47:30.099088 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:30.098410    2580 generic.go:356] "Generic (PLEG): container finished" podID="bc68d92aeaecfd0a0a6d199622a3c375" containerID="5e7c0606ec207717c6eb4425e39a4a991353c53be07fe0a54c1028e18aac140b" exitCode=0
Sep 07 11:47:30.099088 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:30.098731    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal" event={"ID":"bc68d92aeaecfd0a0a6d199622a3c375","Type":"ContainerDied","Data":"5e7c0606ec207717c6eb4425e39a4a991353c53be07fe0a54c1028e18aac140b"}
Sep 07 11:47:30.110074 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:30.109995    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nn82z" event={"ID":"5250f748-f185-42d5-bcc6-006887052b3c","Type":"ContainerStarted","Data":"870971741ff106952001cd42b518fa6068c3fb67a117f8812b11ccb28f9c693b"}
Sep 07 11:47:30.114928 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:30.114037    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-68.ec2.internal" podStartSLOduration=3.114015024 podStartE2EDuration="3.114015024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:30.05335946 +0000 UTC m=+4.549248884" watchObservedRunningTime="2026-09-07 11:47:30.114015024 +0000 UTC m=+4.609904449"
Sep 07 11:47:31.005385 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:31.004804    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:31.005385 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:31.004935    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:31.149312 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:31.149184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal" event={"ID":"bc68d92aeaecfd0a0a6d199622a3c375","Type":"ContainerStarted","Data":"b74cb01316556d47227b9982fb09b840606e5538e20d6cee0e001eea8aab0d91"}
Sep 07 11:47:31.541902 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:31.541277    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret\") pod \"global-pull-secret-syncer-8s4jc\" (UID: \"ec37e993-169b-4dc5-b5f6-e938f46e33d7\") " pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:31.541902 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:31.541387    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:31.541902 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:31.541453    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret podName:ec37e993-169b-4dc5-b5f6-e938f46e33d7 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:35.5414346 +0000 UTC m=+10.037324024 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret") pod "global-pull-secret-syncer-8s4jc" (UID: "ec37e993-169b-4dc5-b5f6-e938f46e33d7") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:31.642115 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:31.642078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs\") pod \"network-metrics-daemon-4dgj8\" (UID: \"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f\") " pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:31.642314 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:31.642129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkzs7\" (UniqueName: \"kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7\") pod \"network-check-target-q976p\" (UID: \"96cc191d-da2d-4e0e-9995-bde02f99e254\") " pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:31.642378 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:31.642318    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:47:31.642378 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:31.642341    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:47:31.642378 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:31.642354    2580 projected.go:196] Error preparing data for projected volume kube-api-access-dkzs7 for pod openshift-network-diagnostics/network-check-target-q976p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:31.642522 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:31.642416    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7 podName:96cc191d-da2d-4e0e-9995-bde02f99e254 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:35.642394511 +0000 UTC m=+10.138283935 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dkzs7" (UniqueName: "kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7") pod "network-check-target-q976p" (UID: "96cc191d-da2d-4e0e-9995-bde02f99e254") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:31.642865 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:31.642839    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:31.642976 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:31.642898    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs podName:7e4e91b4-23e4-4903-8e2d-cf1d2145b51f nodeName:}" failed. No retries permitted until 2026-09-07 11:47:35.642881889 +0000 UTC m=+10.138771290 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs") pod "network-metrics-daemon-4dgj8" (UID: "7e4e91b4-23e4-4903-8e2d-cf1d2145b51f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:32.005182 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:32.004647    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:32.005182 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:32.004764    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:32.005182 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:32.005051    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:32.005182 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:32.005170    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:47:33.003899 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:33.003869    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:33.004363 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:33.003985    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:34.004996 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:34.004964    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:34.005809 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:34.004964    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:34.005809 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:34.005137    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:34.005809 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:34.005360    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:47:35.004084 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:35.003998    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:35.004265 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:35.004131    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:35.575526 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:35.575489    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret\") pod \"global-pull-secret-syncer-8s4jc\" (UID: \"ec37e993-169b-4dc5-b5f6-e938f46e33d7\") " pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:35.575971 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:35.575654    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:35.575971 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:35.575725    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret podName:ec37e993-169b-4dc5-b5f6-e938f46e33d7 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:43.575704555 +0000 UTC m=+18.071593961 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret") pod "global-pull-secret-syncer-8s4jc" (UID: "ec37e993-169b-4dc5-b5f6-e938f46e33d7") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:35.676746 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:35.676708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs\") pod \"network-metrics-daemon-4dgj8\" (UID: \"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f\") " pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:35.676924 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:35.676758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkzs7\" (UniqueName: \"kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7\") pod \"network-check-target-q976p\" (UID: \"96cc191d-da2d-4e0e-9995-bde02f99e254\") " pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:35.676924 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:35.676872    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:35.676924 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:35.676897    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:47:35.676924 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:35.676917    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:47:35.677114 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:35.676929    2580 projected.go:196] Error preparing data for projected volume kube-api-access-dkzs7 for pod openshift-network-diagnostics/network-check-target-q976p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:35.677114 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:35.676943    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs podName:7e4e91b4-23e4-4903-8e2d-cf1d2145b51f nodeName:}" failed. No retries permitted until 2026-09-07 11:47:43.676923032 +0000 UTC m=+18.172812434 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs") pod "network-metrics-daemon-4dgj8" (UID: "7e4e91b4-23e4-4903-8e2d-cf1d2145b51f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:35.677114 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:35.676970    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7 podName:96cc191d-da2d-4e0e-9995-bde02f99e254 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:43.676958623 +0000 UTC m=+18.172848046 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dkzs7" (UniqueName: "kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7") pod "network-check-target-q976p" (UID: "96cc191d-da2d-4e0e-9995-bde02f99e254") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:36.005734 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:36.005056    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:36.005734 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:36.005167    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:36.005734 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:36.005525    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:36.005734 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:36.005635    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:47:37.004341 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:37.004299    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:37.004906 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:37.004445    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:38.004446 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:38.004408    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:38.004893 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:38.004408    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:38.004893 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:38.004548    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:38.004893 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:38.004596    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:47:39.004786 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:39.004745    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:39.005259 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:39.004854    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:40.004193 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:40.004104    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:40.004193 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:40.004146    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:40.004448 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:40.004257    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:40.004448 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:40.004380    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:47:41.004900 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:41.004865    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:41.005373 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:41.004976    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:42.004085 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:42.004044    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:42.004292 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:42.004183    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:42.004292 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:42.004225    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:42.004679 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:42.004643    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:47:43.004838 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:43.004811    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:43.005136 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:43.004906    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:43.632933 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:43.632888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret\") pod \"global-pull-secret-syncer-8s4jc\" (UID: \"ec37e993-169b-4dc5-b5f6-e938f46e33d7\") " pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:43.633144 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:43.633025    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:43.633144 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:43.633089    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret podName:ec37e993-169b-4dc5-b5f6-e938f46e33d7 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:59.633071489 +0000 UTC m=+34.128960893 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret") pod "global-pull-secret-syncer-8s4jc" (UID: "ec37e993-169b-4dc5-b5f6-e938f46e33d7") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:43.733917 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:43.733882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs\") pod \"network-metrics-daemon-4dgj8\" (UID: \"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f\") " pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:43.733917 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:43.733931    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkzs7\" (UniqueName: \"kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7\") pod \"network-check-target-q976p\" (UID: \"96cc191d-da2d-4e0e-9995-bde02f99e254\") " pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:43.734157 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:43.734092    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:43.734234 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:43.734170    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs podName:7e4e91b4-23e4-4903-8e2d-cf1d2145b51f nodeName:}" failed. No retries permitted until 2026-09-07 11:47:59.734150323 +0000 UTC m=+34.230039737 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs") pod "network-metrics-daemon-4dgj8" (UID: "7e4e91b4-23e4-4903-8e2d-cf1d2145b51f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:43.734234 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:43.734180    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:47:43.734234 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:43.734217    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:47:43.734234 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:43.734232    2580 projected.go:196] Error preparing data for projected volume kube-api-access-dkzs7 for pod openshift-network-diagnostics/network-check-target-q976p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:43.734434 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:43.734294    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7 podName:96cc191d-da2d-4e0e-9995-bde02f99e254 nodeName:}" failed. No retries permitted until 2026-09-07 11:47:59.73427602 +0000 UTC m=+34.230165443 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dkzs7" (UniqueName: "kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7") pod "network-check-target-q976p" (UID: "96cc191d-da2d-4e0e-9995-bde02f99e254") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:44.004693 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:44.004615    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:44.004693 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:44.004634    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:44.004858 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:44.004732    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:44.005121 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:44.004875    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:47:45.004838 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:45.004802    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:45.005274 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:45.004929    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:46.005577 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:46.005539    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:46.006065 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:46.005683    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:46.006065 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:46.005744    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:46.006065 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:46.005887    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:47:47.004373 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.004149    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:47.004513 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:47.004487    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:47.183418 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.183381    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dntlp" event={"ID":"fff6de68-6332-4f28-a0cb-60625f82ac14","Type":"ContainerStarted","Data":"a8e0b60ec2965bf9ad195bd5818713735ab6a530fd5401cb97cf646296a75428"}
Sep 07 11:47:47.184989 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.184956    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hwhqc" event={"ID":"8a42548a-1c35-4dc8-8ddd-8d6777ac544e","Type":"ContainerStarted","Data":"e225ff68610b4805159ddc2259085da4d82baa392ef7fbd0ab2fbe02bf55e747"}
Sep 07 11:47:47.186662 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.186638    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554" event={"ID":"c7a26d7e-e575-483f-ae74-ab2a1959e8d9","Type":"ContainerStarted","Data":"463f6cee9937f1a9190eb58d764bf27aadc32204063a89294a35cb5b6b61fe14"}
Sep 07 11:47:47.188416 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.188387    2580 generic.go:356] "Generic (PLEG): container finished" podID="a8041043-f46e-4a5f-8e1f-01018a5189bb" containerID="ede4e44837becb4dded9e9bb67d54da873ee5e555217851b025f194b1d41c7b7" exitCode=0
Sep 07 11:47:47.188522 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.188429    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4fj7g" event={"ID":"a8041043-f46e-4a5f-8e1f-01018a5189bb","Type":"ContainerDied","Data":"ede4e44837becb4dded9e9bb67d54da873ee5e555217851b025f194b1d41c7b7"}
Sep 07 11:47:47.191342 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.191286    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47" event={"ID":"ec0a84de-962e-47ab-9639-fc0d09e06b5d","Type":"ContainerStarted","Data":"78aa1c03c8d218dfe3943b3acdd31a6d031be554f0eaddbce623b94c6ac4d1a9"}
Sep 07 11:47:47.191410 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.191359    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47" event={"ID":"ec0a84de-962e-47ab-9639-fc0d09e06b5d","Type":"ContainerStarted","Data":"95fb270abd4b235ebdca0b04a184fa51582a0a481aff23da0d375729907642b9"}
Sep 07 11:47:47.191501 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.191414    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47" event={"ID":"ec0a84de-962e-47ab-9639-fc0d09e06b5d","Type":"ContainerStarted","Data":"3504dd8e975283964cd3344b8cb05392fe623757f122aeb7438110f26548e2dc"}
Sep 07 11:47:47.191501 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.191449    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47" event={"ID":"ec0a84de-962e-47ab-9639-fc0d09e06b5d","Type":"ContainerStarted","Data":"9b8d5c14b880475094e9c523a7ec62f0638eb077dcc2f7d7515518def5cec113"}
Sep 07 11:47:47.192885 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.192862    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kwr4n" event={"ID":"3c2e42f2-bf8f-4093-b496-af2041e0090e","Type":"ContainerStarted","Data":"b6b0c082d0a329be259f514116b2c6355545b1b62d9f8a399adfccecd46dba38"}
Sep 07 11:47:47.194475 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.194455    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9f27p" event={"ID":"8722b24c-1b27-4443-aacb-b96b30a04bf3","Type":"ContainerStarted","Data":"b1b2984cb09e27068bc2d89824cbeab3c3261d80f8de2fcef9d2275ac254876a"}
Sep 07 11:47:47.195958 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.195938    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mq4kn" event={"ID":"7300402b-f9fd-4f08-8728-b3a4ad8e2d8e","Type":"ContainerStarted","Data":"b7ee26596179428537c2ffc54b26dd491bcaae8bfcc8abb798e899813b4b0f63"}
Sep 07 11:47:47.200708 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.200671    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dntlp" podStartSLOduration=3.7134517049999998 podStartE2EDuration="21.200659825s" totalImagesPullingTime="17.48720812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:47.200127677 +0000 UTC m=+21.696017099" watchObservedRunningTime="2026-09-07 11:47:47.200659825 +0000 UTC m=+21.696549246"
Sep 07 11:47:47.200834 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.200793    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-68.ec2.internal" podStartSLOduration=20.200786929 podStartE2EDuration="20.200786929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:31.163343649 +0000 UTC m=+5.659233073" watchObservedRunningTime="2026-09-07 11:47:47.200786929 +0000 UTC m=+21.696676353"
Sep 07 11:47:47.230516 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.230452    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9f27p" podStartSLOduration=3.754000399 podStartE2EDuration="21.230431774s" totalImagesPullingTime="17.476431375s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:47.21313415 +0000 UTC m=+21.709023574" watchObservedRunningTime="2026-09-07 11:47:47.230431774 +0000 UTC m=+21.726321199"
Sep 07 11:47:47.230862 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.230834    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mq4kn" podStartSLOduration=3.73250157 podStartE2EDuration="21.230825466s" totalImagesPullingTime="17.498323896s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:47.230585982 +0000 UTC m=+21.726475397" watchObservedRunningTime="2026-09-07 11:47:47.230825466 +0000 UTC m=+21.726714888"
Sep 07 11:47:47.245597 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.245559    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kwr4n" podStartSLOduration=3.767434433 podStartE2EDuration="21.24554528s" totalImagesPullingTime="17.478110847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:47.244938953 +0000 UTC m=+21.740828379" watchObservedRunningTime="2026-09-07 11:47:47.24554528 +0000 UTC m=+21.741434703"
Sep 07 11:47:47.281869 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.281829    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hwhqc" podStartSLOduration=7.144024643 podStartE2EDuration="21.281814957s" totalImagesPullingTime="14.137790314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:47.281764167 +0000 UTC m=+21.777653590" watchObservedRunningTime="2026-09-07 11:47:47.281814957 +0000 UTC m=+21.777704379"
Sep 07 11:47:47.556807 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.556784    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hwhqc"
Sep 07 11:47:47.557360 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:47.557343    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hwhqc"
Sep 07 11:47:48.004976 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:48.004925    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:48.005221 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:48.005058    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:48.005221 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:48.005095    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:48.005221 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:48.005209    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:47:48.014695 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:48.014665    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 07 11:47:48.201157 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:48.201124    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554" event={"ID":"c7a26d7e-e575-483f-ae74-ab2a1959e8d9","Type":"ContainerStarted","Data":"4862f5c6207f98b307331deca1ac1ffabb575473d6af8ce04b0d76ab9f06a8a3"}
Sep 07 11:47:48.203913 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:48.203881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47" event={"ID":"ec0a84de-962e-47ab-9639-fc0d09e06b5d","Type":"ContainerStarted","Data":"23996268d685d00376f47c33d68945e83dd389880ad46eb9a832bfd7440eebb9"}
Sep 07 11:47:48.204021 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:48.203948    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47" event={"ID":"ec0a84de-962e-47ab-9639-fc0d09e06b5d","Type":"ContainerStarted","Data":"0b3cb7714fb28b30e12633d2288d7ac1a89817a1a2c9e735b95dd20897364282"}
Sep 07 11:47:48.730672 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:48.730627    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hwhqc"
Sep 07 11:47:48.731363 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:48.731341    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hwhqc"
Sep 07 11:47:48.950550 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:48.950425    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-07T11:47:48.014690294Z","UUID":"c13c64a7-c3e1-42c7-b75f-99e90032a5de","Handler":null,"Name":"","Endpoint":""}
Sep 07 11:47:48.952019 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:48.952000    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 07 11:47:48.952141 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:48.952035    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 07 11:47:49.004807 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:49.004732    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:49.004948 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:49.004850    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:49.208559 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:49.208528    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nn82z" event={"ID":"5250f748-f185-42d5-bcc6-006887052b3c","Type":"ContainerStarted","Data":"c40de30285ef557120395da516068172e85e697ea87b069bc8fc1183229581cc"}
Sep 07 11:47:49.211193 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:49.211159    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554" event={"ID":"c7a26d7e-e575-483f-ae74-ab2a1959e8d9","Type":"ContainerStarted","Data":"5b5663abba7398ed4482176854a2c6cfaf1f905630c1085a4220c4b0a53c52a2"}
Sep 07 11:47:49.223447 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:49.223401    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nn82z" podStartSLOduration=5.7533116060000005 podStartE2EDuration="23.223387119s" totalImagesPullingTime="17.470075513s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:49.223172854 +0000 UTC m=+23.719062277" watchObservedRunningTime="2026-09-07 11:47:49.223387119 +0000 UTC m=+23.719276540"
Sep 07 11:47:50.004284 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:50.004248    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:50.004472 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:50.004290    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:50.004472 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:50.004382    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:50.004588 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:50.004533    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:47:50.219356 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:50.219317    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47" event={"ID":"ec0a84de-962e-47ab-9639-fc0d09e06b5d","Type":"ContainerStarted","Data":"186e9b278125f1be96dcd51db114f37b082d0c480f23c0ac1a75519157bca609"}
Sep 07 11:47:51.004385 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:51.004347    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:51.004574 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:51.004476    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:52.004474 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:52.004292    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:52.005101 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:52.004292    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:52.005101 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:52.004570    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:52.005101 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:52.004623    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:47:52.225917 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:52.225888    2580 generic.go:356] "Generic (PLEG): container finished" podID="a8041043-f46e-4a5f-8e1f-01018a5189bb" containerID="17022fd993754328952c8e138f1a3dcdf0a9d36d169bd678daf7a1a064b33b7f" exitCode=0
Sep 07 11:47:52.226102 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:52.225973    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4fj7g" event={"ID":"a8041043-f46e-4a5f-8e1f-01018a5189bb","Type":"ContainerDied","Data":"17022fd993754328952c8e138f1a3dcdf0a9d36d169bd678daf7a1a064b33b7f"}
Sep 07 11:47:52.229623 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:52.229590    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47" event={"ID":"ec0a84de-962e-47ab-9639-fc0d09e06b5d","Type":"ContainerStarted","Data":"489b2feb9bb2b840f48a95c17a9b1ffc01ad9faa928869e16e9387e9a4663d1c"}
Sep 07 11:47:52.229911 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:52.229893    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:52.244558 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:52.244536    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:52.245879 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:52.245839    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jg554" podStartSLOduration=6.591665473 podStartE2EDuration="26.245825972s" totalImagesPullingTime="19.654160499s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:49.243924636 +0000 UTC m=+23.739814059" watchObservedRunningTime="2026-09-07 11:47:52.245825972 +0000 UTC m=+26.741715448"
Sep 07 11:47:52.268694 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:52.268645    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47" podStartSLOduration=8.575878805 podStartE2EDuration="26.268629623s" totalImagesPullingTime="17.692750818s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:47:52.26817927 +0000 UTC m=+26.764068694" watchObservedRunningTime="2026-09-07 11:47:52.268629623 +0000 UTC m=+26.764519045"
Sep 07 11:47:53.004637 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:53.004601    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:53.005191 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:53.004726    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:53.234349 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:53.233125    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 07 11:47:53.235321 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:53.234763    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:53.252363 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:53.252336    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:53.670151 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:53.669899    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8s4jc"]
Sep 07 11:47:53.670318 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:53.670292    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:53.670433 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:53.670410    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:53.672456 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:53.672434    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4dgj8"]
Sep 07 11:47:53.672569 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:53.672524    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:53.672645 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:53.672626    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:47:53.682800 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:53.682773    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q976p"]
Sep 07 11:47:53.682928 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:53.682881    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:53.682965 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:53.682949    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:54.237453 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:54.237421    2580 generic.go:356] "Generic (PLEG): container finished" podID="a8041043-f46e-4a5f-8e1f-01018a5189bb" containerID="1d896f3a030b72c965270683cea3e14c3e2497620340e228d7d23e6975f2112f" exitCode=0
Sep 07 11:47:54.237887 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:54.237494    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4fj7g" event={"ID":"a8041043-f46e-4a5f-8e1f-01018a5189bb","Type":"ContainerDied","Data":"1d896f3a030b72c965270683cea3e14c3e2497620340e228d7d23e6975f2112f"}
Sep 07 11:47:54.237887 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:54.237560    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 07 11:47:55.004105 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:55.004077    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:55.004274 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:55.004188    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:55.196683 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:55.196652    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:47:56.004820 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:56.004752    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:56.005139 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:56.004849    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:56.005139 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:56.004895    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:56.005139 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:56.004986    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:47:56.243817 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:56.243782    2580 generic.go:356] "Generic (PLEG): container finished" podID="a8041043-f46e-4a5f-8e1f-01018a5189bb" containerID="dd4dd18abe3e1a899cbd0c25d67390e29ea0f90f5d47cc2900ada394c6d547d6" exitCode=0
Sep 07 11:47:56.243988 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:56.243858    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4fj7g" event={"ID":"a8041043-f46e-4a5f-8e1f-01018a5189bb","Type":"ContainerDied","Data":"dd4dd18abe3e1a899cbd0c25d67390e29ea0f90f5d47cc2900ada394c6d547d6"}
Sep 07 11:47:56.257250 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:56.257038    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47" podUID="ec0a84de-962e-47ab-9639-fc0d09e06b5d" containerName="ovnkube-controller" probeResult="failure" output=""
Sep 07 11:47:57.004846 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:57.004817    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:57.005320 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:57.004909    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:58.004126 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:58.004090    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:58.004312 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:58.004105    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:58.004312 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:58.004286    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:47:58.004475 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:58.004450    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:47:59.004576 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:59.004537    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:59.004977 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:59.004647    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8s4jc" podUID="ec37e993-169b-4dc5-b5f6-e938f46e33d7"
Sep 07 11:47:59.646177 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:59.646143    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret\") pod \"global-pull-secret-syncer-8s4jc\" (UID: \"ec37e993-169b-4dc5-b5f6-e938f46e33d7\") " pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:47:59.646356 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:59.646262    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:59.646356 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:59.646313    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret podName:ec37e993-169b-4dc5-b5f6-e938f46e33d7 nodeName:}" failed. No retries permitted until 2026-09-07 11:48:31.646300706 +0000 UTC m=+66.142190107 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret") pod "global-pull-secret-syncer-8s4jc" (UID: "ec37e993-169b-4dc5-b5f6-e938f46e33d7") : object "kube-system"/"original-pull-secret" not registered
Sep 07 11:47:59.746642 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:59.746609    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs\") pod \"network-metrics-daemon-4dgj8\" (UID: \"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f\") " pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:47:59.746642 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:47:59.746648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkzs7\" (UniqueName: \"kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7\") pod \"network-check-target-q976p\" (UID: \"96cc191d-da2d-4e0e-9995-bde02f99e254\") " pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:47:59.746858 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:59.746781    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 11:47:59.746858 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:59.746800    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 11:47:59.746858 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:59.746809    2580 projected.go:196] Error preparing data for projected volume kube-api-access-dkzs7 for pod openshift-network-diagnostics/network-check-target-q976p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:59.746858 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:59.746857    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7 podName:96cc191d-da2d-4e0e-9995-bde02f99e254 nodeName:}" failed. No retries permitted until 2026-09-07 11:48:31.746841875 +0000 UTC m=+66.242731276 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dkzs7" (UniqueName: "kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7") pod "network-check-target-q976p" (UID: "96cc191d-da2d-4e0e-9995-bde02f99e254") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 11:47:59.747025 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:59.746782    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:47:59.747025 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:47:59.746940    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs podName:7e4e91b4-23e4-4903-8e2d-cf1d2145b51f nodeName:}" failed. No retries permitted until 2026-09-07 11:48:31.746920642 +0000 UTC m=+66.242810059 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs") pod "network-metrics-daemon-4dgj8" (UID: "7e4e91b4-23e4-4903-8e2d-cf1d2145b51f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 11:48:00.007956 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.007881    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:48:00.008458 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:00.008015    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:48:00.008458 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.008082    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:48:00.008458 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:00.008187    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q976p" podUID="96cc191d-da2d-4e0e-9995-bde02f99e254"
Sep 07 11:48:00.259980 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.259930    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-68.ec2.internal" event="NodeReady"
Sep 07 11:48:00.260092 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.260080    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 07 11:48:00.305686 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.305638    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jsx9s"]
Sep 07 11:48:00.308547 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.308528    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vb6cl"]
Sep 07 11:48:00.308656 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.308641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:00.311032 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.311010    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 07 11:48:00.311126 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.311044    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 07 11:48:00.311126 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.311044    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kd29r\""
Sep 07 11:48:00.311420 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.311404    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vb6cl"
Sep 07 11:48:00.313744 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.313722    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 07 11:48:00.313846 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.313750    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7tkzl\""
Sep 07 11:48:00.313846 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.313759    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 07 11:48:00.313846 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.313725    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 07 11:48:00.316278 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.316256    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vb6cl"]
Sep 07 11:48:00.320127 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.320103    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jsx9s"]
Sep 07 11:48:00.452169 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.452142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzwj7\" (UniqueName: \"kubernetes.io/projected/17691127-7e46-44e2-a052-edf0873d6f26-kube-api-access-hzwj7\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:00.452293 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.452177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02120513-1881-4f03-89fb-c94109d599b4-cert\") pod \"ingress-canary-vb6cl\" (UID: \"02120513-1881-4f03-89fb-c94109d599b4\") " pod="openshift-ingress-canary/ingress-canary-vb6cl"
Sep 07 11:48:00.452293 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.452217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/17691127-7e46-44e2-a052-edf0873d6f26-config-volume\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:00.455063 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.452237    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:00.455063 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.452767    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17691127-7e46-44e2-a052-edf0873d6f26-tmp-dir\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:00.455063 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.452867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdtth\" (UniqueName: \"kubernetes.io/projected/02120513-1881-4f03-89fb-c94109d599b4-kube-api-access-sdtth\") pod \"ingress-canary-vb6cl\" (UID: \"02120513-1881-4f03-89fb-c94109d599b4\") " pod="openshift-ingress-canary/ingress-canary-vb6cl"
Sep 07 11:48:00.554221 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.554099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzwj7\" (UniqueName: \"kubernetes.io/projected/17691127-7e46-44e2-a052-edf0873d6f26-kube-api-access-hzwj7\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:00.554221 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.554147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02120513-1881-4f03-89fb-c94109d599b4-cert\") pod \"ingress-canary-vb6cl\" (UID: \"02120513-1881-4f03-89fb-c94109d599b4\") " pod="openshift-ingress-canary/ingress-canary-vb6cl"
Sep 07 11:48:00.554457 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.554275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/17691127-7e46-44e2-a052-edf0873d6f26-config-volume\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:00.554457 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.554308    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:00.554457 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.554337    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17691127-7e46-44e2-a052-edf0873d6f26-tmp-dir\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:00.554457 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.554376    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdtth\" (UniqueName: \"kubernetes.io/projected/02120513-1881-4f03-89fb-c94109d599b4-kube-api-access-sdtth\") pod \"ingress-canary-vb6cl\" (UID: \"02120513-1881-4f03-89fb-c94109d599b4\") " pod="openshift-ingress-canary/ingress-canary-vb6cl"
Sep 07 11:48:00.554615 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:00.554476    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:48:00.554615 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:00.554545    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls podName:17691127-7e46-44e2-a052-edf0873d6f26 nodeName:}" failed. No retries permitted until 2026-09-07 11:48:01.054528153 +0000 UTC m=+35.550417573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls") pod "dns-default-jsx9s" (UID: "17691127-7e46-44e2-a052-edf0873d6f26") : secret "dns-default-metrics-tls" not found
Sep 07 11:48:00.554750 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.554730    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17691127-7e46-44e2-a052-edf0873d6f26-tmp-dir\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:00.554893 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.554873    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/17691127-7e46-44e2-a052-edf0873d6f26-config-volume\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:00.558246 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.558225    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02120513-1881-4f03-89fb-c94109d599b4-cert\") pod \"ingress-canary-vb6cl\" (UID: \"02120513-1881-4f03-89fb-c94109d599b4\") " pod="openshift-ingress-canary/ingress-canary-vb6cl"
Sep 07 11:48:00.562416 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.562395    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdtth\" (UniqueName: \"kubernetes.io/projected/02120513-1881-4f03-89fb-c94109d599b4-kube-api-access-sdtth\") pod \"ingress-canary-vb6cl\" (UID: \"02120513-1881-4f03-89fb-c94109d599b4\") " pod="openshift-ingress-canary/ingress-canary-vb6cl"
Sep 07 11:48:00.562490 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.562427    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzwj7\" (UniqueName: \"kubernetes.io/projected/17691127-7e46-44e2-a052-edf0873d6f26-kube-api-access-hzwj7\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:00.626183 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.626136    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vb6cl"
Sep 07 11:48:00.901390 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:00.901359    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vb6cl"]
Sep 07 11:48:00.905609 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:48:00.905569    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02120513_1881_4f03_89fb_c94109d599b4.slice/crio-8ce0135b3e864b1551d3b31ea1257b3dab73a3bd37df060a8ba43e2331e85ad1 WatchSource:0}: Error finding container 8ce0135b3e864b1551d3b31ea1257b3dab73a3bd37df060a8ba43e2331e85ad1: Status 404 returned error can't find the container with id 8ce0135b3e864b1551d3b31ea1257b3dab73a3bd37df060a8ba43e2331e85ad1
Sep 07 11:48:01.004223 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:01.004170    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:48:01.006882 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:01.006838    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 07 11:48:01.058467 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:01.058428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:01.059103 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:01.058574    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:48:01.059103 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:01.058635    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls podName:17691127-7e46-44e2-a052-edf0873d6f26 nodeName:}" failed. No retries permitted until 2026-09-07 11:48:02.058619894 +0000 UTC m=+36.554509298 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls") pod "dns-default-jsx9s" (UID: "17691127-7e46-44e2-a052-edf0873d6f26") : secret "dns-default-metrics-tls" not found
Sep 07 11:48:01.258235 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:01.258099    2580 generic.go:356] "Generic (PLEG): container finished" podID="a8041043-f46e-4a5f-8e1f-01018a5189bb" containerID="0467c96be5f3f8d0292b5ba577f2969daa39555f21e60be179c37e09e4656e89" exitCode=0
Sep 07 11:48:01.258235 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:01.258162    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4fj7g" event={"ID":"a8041043-f46e-4a5f-8e1f-01018a5189bb","Type":"ContainerDied","Data":"0467c96be5f3f8d0292b5ba577f2969daa39555f21e60be179c37e09e4656e89"}
Sep 07 11:48:01.259395 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:01.259375    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vb6cl" event={"ID":"02120513-1881-4f03-89fb-c94109d599b4","Type":"ContainerStarted","Data":"8ce0135b3e864b1551d3b31ea1257b3dab73a3bd37df060a8ba43e2331e85ad1"}
Sep 07 11:48:02.004859 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:02.004752    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:48:02.005052 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:02.005016    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:48:02.007402 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:02.007377    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2v7c7\""
Sep 07 11:48:02.008510 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:02.008473    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 07 11:48:02.008510 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:02.008509    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 07 11:48:02.008743 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:02.008563    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gtblx\""
Sep 07 11:48:02.008743 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:02.008605    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 07 11:48:02.066952 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:02.066921    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:02.067429 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:02.067088    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:48:02.067429 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:02.067166    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls podName:17691127-7e46-44e2-a052-edf0873d6f26 nodeName:}" failed. No retries permitted until 2026-09-07 11:48:04.067145129 +0000 UTC m=+38.563034535 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls") pod "dns-default-jsx9s" (UID: "17691127-7e46-44e2-a052-edf0873d6f26") : secret "dns-default-metrics-tls" not found
Sep 07 11:48:02.267421 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:02.267344    2580 generic.go:356] "Generic (PLEG): container finished" podID="a8041043-f46e-4a5f-8e1f-01018a5189bb" containerID="14bc0519509f421c4beb186f039ca2077658ba93182334754da6eba435d30597" exitCode=0
Sep 07 11:48:02.267563 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:02.267424    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4fj7g" event={"ID":"a8041043-f46e-4a5f-8e1f-01018a5189bb","Type":"ContainerDied","Data":"14bc0519509f421c4beb186f039ca2077658ba93182334754da6eba435d30597"}
Sep 07 11:48:03.274275 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:03.274193    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4fj7g" event={"ID":"a8041043-f46e-4a5f-8e1f-01018a5189bb","Type":"ContainerStarted","Data":"5f13c14df630a14ac9a316d4f111f0d6c76e15f5b566822bac3b2997f22a6f0c"}
Sep 07 11:48:03.308540 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:03.308475    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4fj7g" podStartSLOduration=9.067348563 podStartE2EDuration="37.30845648s" totalImagesPullingTime="28.241107917s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:48:03.306393634 +0000 UTC m=+37.802283103" watchObservedRunningTime="2026-09-07 11:48:03.30845648 +0000 UTC m=+37.804345902"
Sep 07 11:48:04.081020 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:04.080835    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:04.081168 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:04.080977    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:48:04.081168 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:04.081077    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls podName:17691127-7e46-44e2-a052-edf0873d6f26 nodeName:}" failed. No retries permitted until 2026-09-07 11:48:08.081063016 +0000 UTC m=+42.576952416 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls") pod "dns-default-jsx9s" (UID: "17691127-7e46-44e2-a052-edf0873d6f26") : secret "dns-default-metrics-tls" not found
Sep 07 11:48:04.277708 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:04.277646    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vb6cl" event={"ID":"02120513-1881-4f03-89fb-c94109d599b4","Type":"ContainerStarted","Data":"a164771dd98560cd1c5f2a82b254ca741e01d5b407abe226fb59120243a28419"}
Sep 07 11:48:04.292231 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:04.292174    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vb6cl" podStartSLOduration=1.712995627 podStartE2EDuration="4.292159597s" totalImagesPullingTime="2.57916397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:48:04.291162682 +0000 UTC m=+38.787052085" watchObservedRunningTime="2026-09-07 11:48:04.292159597 +0000 UTC m=+38.788049019"
Sep 07 11:48:08.108162 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:08.108121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:08.108549 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:08.108282    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:48:08.108549 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:08.108339    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls podName:17691127-7e46-44e2-a052-edf0873d6f26 nodeName:}" failed. No retries permitted until 2026-09-07 11:48:16.108322989 +0000 UTC m=+50.604212405 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls") pod "dns-default-jsx9s" (UID: "17691127-7e46-44e2-a052-edf0873d6f26") : secret "dns-default-metrics-tls" not found
Sep 07 11:48:16.163221 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:16.163160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:16.163636 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:16.163312    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:48:16.163636 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:16.163373    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls podName:17691127-7e46-44e2-a052-edf0873d6f26 nodeName:}" failed. No retries permitted until 2026-09-07 11:48:32.163358068 +0000 UTC m=+66.659247469 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls") pod "dns-default-jsx9s" (UID: "17691127-7e46-44e2-a052-edf0873d6f26") : secret "dns-default-metrics-tls" not found
Sep 07 11:48:26.253528 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:26.253498    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tjx47"
Sep 07 11:48:31.668678 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:31.668640    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret\") pod \"global-pull-secret-syncer-8s4jc\" (UID: \"ec37e993-169b-4dc5-b5f6-e938f46e33d7\") " pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:48:31.671611 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:31.671589    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 07 11:48:31.683050 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:31.683028    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ec37e993-169b-4dc5-b5f6-e938f46e33d7-original-pull-secret\") pod \"global-pull-secret-syncer-8s4jc\" (UID: \"ec37e993-169b-4dc5-b5f6-e938f46e33d7\") " pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:48:31.769506 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:31.769472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs\") pod \"network-metrics-daemon-4dgj8\" (UID: \"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f\") " pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:48:31.769506 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:31.769511    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkzs7\" (UniqueName: \"kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7\") pod \"network-check-target-q976p\" (UID: \"96cc191d-da2d-4e0e-9995-bde02f99e254\") " pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:48:31.772341 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:31.772322    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 07 11:48:31.772341 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:31.772335    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 07 11:48:31.780318 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:31.780300    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 07 11:48:31.780384 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:31.780356    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs podName:7e4e91b4-23e4-4903-8e2d-cf1d2145b51f nodeName:}" failed. No retries permitted until 2026-09-07 11:49:35.780339344 +0000 UTC m=+130.276228749 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs") pod "network-metrics-daemon-4dgj8" (UID: "7e4e91b4-23e4-4903-8e2d-cf1d2145b51f") : secret "metrics-daemon-secret" not found
Sep 07 11:48:31.782563 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:31.782545    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 07 11:48:31.793139 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:31.793109    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkzs7\" (UniqueName: \"kubernetes.io/projected/96cc191d-da2d-4e0e-9995-bde02f99e254-kube-api-access-dkzs7\") pod \"network-check-target-q976p\" (UID: \"96cc191d-da2d-4e0e-9995-bde02f99e254\") " pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:48:31.913629 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:31.913591    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8s4jc"
Sep 07 11:48:32.018483 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:32.018452    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2v7c7\""
Sep 07 11:48:32.026586 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:32.026556    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:48:32.172793 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:32.172759    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:48:32.172926 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:32.172903    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:48:32.172983 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:48:32.172966    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls podName:17691127-7e46-44e2-a052-edf0873d6f26 nodeName:}" failed. No retries permitted until 2026-09-07 11:49:04.172950552 +0000 UTC m=+98.668839952 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls") pod "dns-default-jsx9s" (UID: "17691127-7e46-44e2-a052-edf0873d6f26") : secret "dns-default-metrics-tls" not found
Sep 07 11:48:32.210173 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:32.210093    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8s4jc"]
Sep 07 11:48:32.213028 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:48:32.213004    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec37e993_169b_4dc5_b5f6_e938f46e33d7.slice/crio-7ef180b54553c793fdeb71b0e33f58f90acdb042b19f951c38ffc3915cd8752d WatchSource:0}: Error finding container 7ef180b54553c793fdeb71b0e33f58f90acdb042b19f951c38ffc3915cd8752d: Status 404 returned error can't find the container with id 7ef180b54553c793fdeb71b0e33f58f90acdb042b19f951c38ffc3915cd8752d
Sep 07 11:48:32.306750 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:32.306715    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q976p"]
Sep 07 11:48:32.309942 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:48:32.309916    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96cc191d_da2d_4e0e_9995_bde02f99e254.slice/crio-c85b86fa7d67db89c82e3b3535d397324feb5f3e1a2ce74005148148c88fb641 WatchSource:0}: Error finding container c85b86fa7d67db89c82e3b3535d397324feb5f3e1a2ce74005148148c88fb641: Status 404 returned error can't find the container with id c85b86fa7d67db89c82e3b3535d397324feb5f3e1a2ce74005148148c88fb641
Sep 07 11:48:32.340735 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:32.340712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q976p" event={"ID":"96cc191d-da2d-4e0e-9995-bde02f99e254","Type":"ContainerStarted","Data":"c85b86fa7d67db89c82e3b3535d397324feb5f3e1a2ce74005148148c88fb641"}
Sep 07 11:48:32.341760 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:32.341738    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8s4jc" event={"ID":"ec37e993-169b-4dc5-b5f6-e938f46e33d7","Type":"ContainerStarted","Data":"7ef180b54553c793fdeb71b0e33f58f90acdb042b19f951c38ffc3915cd8752d"}
Sep 07 11:48:37.356891 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.356855    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q976p" event={"ID":"96cc191d-da2d-4e0e-9995-bde02f99e254","Type":"ContainerStarted","Data":"8c62ffedc157a4ccf62841bc2e8658c112d2bace1d9d859c922d36bcc968700d"}
Sep 07 11:48:37.357421 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.356978    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:48:37.358452 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.358432    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8s4jc" event={"ID":"ec37e993-169b-4dc5-b5f6-e938f46e33d7","Type":"ContainerStarted","Data":"fdc25d350aa6fb8ee67941443bedda6173c2fde6f0c0fb125f90a714f70c6ab7"}
Sep 07 11:48:37.372069 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.372025    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-q976p" podStartSLOduration=67.08167112 podStartE2EDuration="1m11.372012424s" totalImagesPullingTime="4.290341304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:48:37.371379387 +0000 UTC m=+71.867268811" watchObservedRunningTime="2026-09-07 11:48:37.372012424 +0000 UTC m=+71.867901846"
Sep 07 11:48:37.384984 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.384941    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8s4jc" podStartSLOduration=66.994371484 podStartE2EDuration="1m11.384927913s" totalImagesPullingTime="4.390556429s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:48:37.38440418 +0000 UTC m=+71.880293602" watchObservedRunningTime="2026-09-07 11:48:37.384927913 +0000 UTC m=+71.880817335"
Sep 07 11:48:37.808987 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.808949    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"]
Sep 07 11:48:37.812971 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.812954    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"]
Sep 07 11:48:37.813096 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.813078    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:37.815461 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.815439    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 07 11:48:37.815942 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.815775    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"
Sep 07 11:48:37.815942 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.815802    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 07 11:48:37.815942 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.815839    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 07 11:48:37.815942 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.815847    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 07 11:48:37.815942 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.815784    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 07 11:48:37.815942 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.815844    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 07 11:48:37.816461 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.816446    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 07 11:48:37.817960 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.817943    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 07 11:48:37.820900 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.820882    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"]
Sep 07 11:48:37.823701 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.823682    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"]
Sep 07 11:48:37.907328 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.907289    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/dce4a004-6319-43a2-ac7e-ed7aafc22569-ca\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:37.907328 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.907336    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/dce4a004-6319-43a2-ac7e-ed7aafc22569-hub\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:37.907548 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.907361    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/dce4a004-6319-43a2-ac7e-ed7aafc22569-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:37.907548 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.907389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/548e4328-97e2-4e66-b392-377d49fa4d1d-klusterlet-config\") pod \"klusterlet-addon-workmgr-7bb4c79995-j4zl2\" (UID: \"548e4328-97e2-4e66-b392-377d49fa4d1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"
Sep 07 11:48:37.907548 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.907406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fzq8\" (UniqueName: \"kubernetes.io/projected/548e4328-97e2-4e66-b392-377d49fa4d1d-kube-api-access-6fzq8\") pod \"klusterlet-addon-workmgr-7bb4c79995-j4zl2\" (UID: \"548e4328-97e2-4e66-b392-377d49fa4d1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"
Sep 07 11:48:37.907548 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.907436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/dce4a004-6319-43a2-ac7e-ed7aafc22569-additional-service-ca\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:37.907548 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.907500    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dce4a004-6319-43a2-ac7e-ed7aafc22569-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:37.907548 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.907526    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/548e4328-97e2-4e66-b392-377d49fa4d1d-tmp\") pod \"klusterlet-addon-workmgr-7bb4c79995-j4zl2\" (UID: \"548e4328-97e2-4e66-b392-377d49fa4d1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"
Sep 07 11:48:37.907775 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:37.907550    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtbjf\" (UniqueName: \"kubernetes.io/projected/dce4a004-6319-43a2-ac7e-ed7aafc22569-kube-api-access-xtbjf\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:38.008101 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.008074    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtbjf\" (UniqueName: \"kubernetes.io/projected/dce4a004-6319-43a2-ac7e-ed7aafc22569-kube-api-access-xtbjf\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:38.008280 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.008109    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/dce4a004-6319-43a2-ac7e-ed7aafc22569-ca\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:38.008280 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.008131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/dce4a004-6319-43a2-ac7e-ed7aafc22569-hub\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:38.008280 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.008147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/dce4a004-6319-43a2-ac7e-ed7aafc22569-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:38.008280 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.008212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/548e4328-97e2-4e66-b392-377d49fa4d1d-klusterlet-config\") pod \"klusterlet-addon-workmgr-7bb4c79995-j4zl2\" (UID: \"548e4328-97e2-4e66-b392-377d49fa4d1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"
Sep 07 11:48:38.008280 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.008247    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fzq8\" (UniqueName: \"kubernetes.io/projected/548e4328-97e2-4e66-b392-377d49fa4d1d-kube-api-access-6fzq8\") pod \"klusterlet-addon-workmgr-7bb4c79995-j4zl2\" (UID: \"548e4328-97e2-4e66-b392-377d49fa4d1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"
Sep 07 11:48:38.008533 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.008384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/dce4a004-6319-43a2-ac7e-ed7aafc22569-additional-service-ca\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:38.008533 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.008452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dce4a004-6319-43a2-ac7e-ed7aafc22569-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:38.008533 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.008481    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/548e4328-97e2-4e66-b392-377d49fa4d1d-tmp\") pod \"klusterlet-addon-workmgr-7bb4c79995-j4zl2\" (UID: \"548e4328-97e2-4e66-b392-377d49fa4d1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"
Sep 07 11:48:38.008871 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.008849    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/548e4328-97e2-4e66-b392-377d49fa4d1d-tmp\") pod \"klusterlet-addon-workmgr-7bb4c79995-j4zl2\" (UID: \"548e4328-97e2-4e66-b392-377d49fa4d1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"
Sep 07 11:48:38.009130 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.009109    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/dce4a004-6319-43a2-ac7e-ed7aafc22569-additional-service-ca\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:38.010991 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.010964    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/548e4328-97e2-4e66-b392-377d49fa4d1d-klusterlet-config\") pod \"klusterlet-addon-workmgr-7bb4c79995-j4zl2\" (UID: \"548e4328-97e2-4e66-b392-377d49fa4d1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"
Sep 07 11:48:38.011244 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.011191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/dce4a004-6319-43a2-ac7e-ed7aafc22569-hub\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:38.011336 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.011316    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/dce4a004-6319-43a2-ac7e-ed7aafc22569-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:38.016019 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.016002    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fzq8\" (UniqueName: \"kubernetes.io/projected/548e4328-97e2-4e66-b392-377d49fa4d1d-kube-api-access-6fzq8\") pod \"klusterlet-addon-workmgr-7bb4c79995-j4zl2\" (UID: \"548e4328-97e2-4e66-b392-377d49fa4d1d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"
Sep 07 11:48:38.020616 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.020580    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/dce4a004-6319-43a2-ac7e-ed7aafc22569-ca\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:38.020714 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.020700    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dce4a004-6319-43a2-ac7e-ed7aafc22569-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:38.022318 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.022300    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtbjf\" (UniqueName: \"kubernetes.io/projected/dce4a004-6319-43a2-ac7e-ed7aafc22569-kube-api-access-xtbjf\") pod \"cluster-proxy-proxy-agent-68d67bfb57-d9d9h\" (UID: \"dce4a004-6319-43a2-ac7e-ed7aafc22569\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:38.135851 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.135799    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:48:38.140842 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.140822    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"
Sep 07 11:48:38.413753 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.413645    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"]
Sep 07 11:48:38.416930 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:38.416904    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"]
Sep 07 11:48:38.418896 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:48:38.418827    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod548e4328_97e2_4e66_b392_377d49fa4d1d.slice/crio-27c113ad0e73ccedc2dd98fb9c44a45e5775ef05895bfa1cddd42e1715be284e WatchSource:0}: Error finding container 27c113ad0e73ccedc2dd98fb9c44a45e5775ef05895bfa1cddd42e1715be284e: Status 404 returned error can't find the container with id 27c113ad0e73ccedc2dd98fb9c44a45e5775ef05895bfa1cddd42e1715be284e
Sep 07 11:48:38.420752 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:48:38.420734    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddce4a004_6319_43a2_ac7e_ed7aafc22569.slice/crio-9fdddd7d0a940d7ce68f0756849e978d4dc5e668e796d8a26a5321759ce0b0f7 WatchSource:0}: Error finding container 9fdddd7d0a940d7ce68f0756849e978d4dc5e668e796d8a26a5321759ce0b0f7: Status 404 returned error can't find the container with id 9fdddd7d0a940d7ce68f0756849e978d4dc5e668e796d8a26a5321759ce0b0f7
Sep 07 11:48:39.366862 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:39.366800    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2" event={"ID":"548e4328-97e2-4e66-b392-377d49fa4d1d","Type":"ContainerStarted","Data":"27c113ad0e73ccedc2dd98fb9c44a45e5775ef05895bfa1cddd42e1715be284e"}
Sep 07 11:48:39.368089 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:39.368060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h" event={"ID":"dce4a004-6319-43a2-ac7e-ed7aafc22569","Type":"ContainerStarted","Data":"9fdddd7d0a940d7ce68f0756849e978d4dc5e668e796d8a26a5321759ce0b0f7"}
Sep 07 11:48:44.382012 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:44.381977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2" event={"ID":"548e4328-97e2-4e66-b392-377d49fa4d1d","Type":"ContainerStarted","Data":"41df3a9fe18e97361f4bbfa1085e0fb96d458a803b7b42fb81e4f304569b54fa"}
Sep 07 11:48:44.382507 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:44.382117    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"
Sep 07 11:48:44.384132 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:44.384111    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"
Sep 07 11:48:44.384393 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:44.384373    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h" event={"ID":"dce4a004-6319-43a2-ac7e-ed7aafc22569","Type":"ContainerStarted","Data":"3a202c85c6d4210cfc2c9a1515aa6ef37f43d023b88644df3e8b6fcd4128744a"}
Sep 07 11:48:44.384484 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:44.384404    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h" event={"ID":"dce4a004-6319-43a2-ac7e-ed7aafc22569","Type":"ContainerStarted","Data":"8ee02b96d5e72dd50bb308b4a5b48f5bbb21ff930a2ca3ac6235c1ba1b6d6514"}
Sep 07 11:48:44.384484 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:44.384417    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h" event={"ID":"dce4a004-6319-43a2-ac7e-ed7aafc22569","Type":"ContainerStarted","Data":"336cf1eb5eb94103f6a654ecba2ce855419c208ec74c50b4079b9bec31b6bc0f"}
Sep 07 11:48:44.399585 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:44.399541    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2" podStartSLOduration=1.829953943 podStartE2EDuration="7.399529423s" totalImagesPullingTime="5.56957548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:48:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:48:44.399142835 +0000 UTC m=+78.895032256" watchObservedRunningTime="2026-09-07 11:48:44.399529423 +0000 UTC m=+78.895418844"
Sep 07 11:48:44.414557 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:48:44.414511    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h" podStartSLOduration=1.848363214 podStartE2EDuration="7.414497049s" totalImagesPullingTime="5.566133835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:48:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:48:44.413792905 +0000 UTC m=+78.909682328" watchObservedRunningTime="2026-09-07 11:48:44.414497049 +0000 UTC m=+78.910386472"
Sep 07 11:49:04.200038 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:49:04.199993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:49:04.200431 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:49:04.200148    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:49:04.200431 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:49:04.200233    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls podName:17691127-7e46-44e2-a052-edf0873d6f26 nodeName:}" failed. No retries permitted until 2026-09-07 11:50:08.200215577 +0000 UTC m=+162.696104991 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls") pod "dns-default-jsx9s" (UID: "17691127-7e46-44e2-a052-edf0873d6f26") : secret "dns-default-metrics-tls" not found
Sep 07 11:49:08.363799 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:49:08.363766    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-q976p"
Sep 07 11:49:35.820935 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:49:35.820880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs\") pod \"network-metrics-daemon-4dgj8\" (UID: \"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f\") " pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:49:35.821365 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:49:35.821024    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 07 11:49:35.821365 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:49:35.821095    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs podName:7e4e91b4-23e4-4903-8e2d-cf1d2145b51f nodeName:}" failed. No retries permitted until 2026-09-07 11:51:37.821080102 +0000 UTC m=+252.316969507 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs") pod "network-metrics-daemon-4dgj8" (UID: "7e4e91b4-23e4-4903-8e2d-cf1d2145b51f") : secret "metrics-daemon-secret" not found
Sep 07 11:50:03.319377 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:50:03.319319    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jsx9s" podUID="17691127-7e46-44e2-a052-edf0873d6f26"
Sep 07 11:50:03.608221 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:50:03.608114    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:50:05.022396 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:50:05.022348    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:50:08.240358 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:50:08.240304    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:50:08.240752 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:50:08.240446    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 11:50:08.240752 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:50:08.240510    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls podName:17691127-7e46-44e2-a052-edf0873d6f26 nodeName:}" failed. No retries permitted until 2026-09-07 11:52:10.240493291 +0000 UTC m=+284.736382692 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls") pod "dns-default-jsx9s" (UID: "17691127-7e46-44e2-a052-edf0873d6f26") : secret "dns-default-metrics-tls" not found
Sep 07 11:50:20.004785 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:50:20.004690    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:50:44.383390 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:50:44.383353    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2" podUID="548e4328-97e2-4e66-b392-377d49fa4d1d" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.9:8000/readyz\": dial tcp 10.134.0.9:8000: connect: connection refused"
Sep 07 11:50:44.729647 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:50:44.729571    2580 generic.go:356] "Generic (PLEG): container finished" podID="548e4328-97e2-4e66-b392-377d49fa4d1d" containerID="41df3a9fe18e97361f4bbfa1085e0fb96d458a803b7b42fb81e4f304569b54fa" exitCode=1
Sep 07 11:50:44.729647 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:50:44.729634    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2" event={"ID":"548e4328-97e2-4e66-b392-377d49fa4d1d","Type":"ContainerDied","Data":"41df3a9fe18e97361f4bbfa1085e0fb96d458a803b7b42fb81e4f304569b54fa"}
Sep 07 11:50:44.729956 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:50:44.729941    2580 scope.go:117] "RemoveContainer" containerID="41df3a9fe18e97361f4bbfa1085e0fb96d458a803b7b42fb81e4f304569b54fa"
Sep 07 11:50:45.734149 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:50:45.734110    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2" event={"ID":"548e4328-97e2-4e66-b392-377d49fa4d1d","Type":"ContainerStarted","Data":"7fec923e5eedcb8f3c08e46f6be4d3a5352bf7a21ba57de4c0724cfd72e86365"}
Sep 07 11:50:45.734542 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:50:45.734429    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"
Sep 07 11:50:45.735014 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:50:45.734995    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7bb4c79995-j4zl2"
Sep 07 11:51:29.432306 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.432272    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zcbwr"]
Sep 07 11:51:29.434144 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.434124    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:29.437422 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.436626    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 07 11:51:29.437422 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.437118    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 07 11:51:29.437422 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.437412    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 07 11:51:29.437707 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.437445    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 07 11:51:29.438648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.438122    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vwz2b\""
Sep 07 11:51:29.445608 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.445583    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zcbwr"]
Sep 07 11:51:29.608152 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.608109    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6f7985a9-98c1-450f-9e35-5401a6574918-crio-socket\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:29.608337 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.608177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f7985a9-98c1-450f-9e35-5401a6574918-data-volume\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:29.608337 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.608229    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:29.608337 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.608249    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sfnk\" (UniqueName: \"kubernetes.io/projected/6f7985a9-98c1-450f-9e35-5401a6574918-kube-api-access-4sfnk\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:29.608337 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.608311    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:29.709576 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.709485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:29.709576 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.709544    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6f7985a9-98c1-450f-9e35-5401a6574918-crio-socket\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:29.709797 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.709599    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f7985a9-98c1-450f-9e35-5401a6574918-data-volume\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:29.709797 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:29.709603    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 11:51:29.709797 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.709625    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:29.709797 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.709649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sfnk\" (UniqueName: \"kubernetes.io/projected/6f7985a9-98c1-450f-9e35-5401a6574918-kube-api-access-4sfnk\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:29.709797 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:29.709674    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm podName:6f7985a9-98c1-450f-9e35-5401a6574918 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:30.20965695 +0000 UTC m=+244.705546351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zcbwr" (UID: "6f7985a9-98c1-450f-9e35-5401a6574918") : configmap "kube-rbac-proxy" not found
Sep 07 11:51:29.709797 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.709677    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6f7985a9-98c1-450f-9e35-5401a6574918-crio-socket\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:29.709797 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:29.709780    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:29.710026 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:29.709831    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls podName:6f7985a9-98c1-450f-9e35-5401a6574918 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:30.209815096 +0000 UTC m=+244.705704516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zcbwr" (UID: "6f7985a9-98c1-450f-9e35-5401a6574918") : secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:29.710026 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.709879    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f7985a9-98c1-450f-9e35-5401a6574918-data-volume\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:29.720640 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:29.720614    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sfnk\" (UniqueName: \"kubernetes.io/projected/6f7985a9-98c1-450f-9e35-5401a6574918-kube-api-access-4sfnk\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:30.214493 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:30.214449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:30.214666 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:30.214509    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:30.214666 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:30.214594    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 11:51:30.214666 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:30.214612    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:30.214666 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:30.214657    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm podName:6f7985a9-98c1-450f-9e35-5401a6574918 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:31.214640959 +0000 UTC m=+245.710530359 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zcbwr" (UID: "6f7985a9-98c1-450f-9e35-5401a6574918") : configmap "kube-rbac-proxy" not found
Sep 07 11:51:30.214804 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:30.214677    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls podName:6f7985a9-98c1-450f-9e35-5401a6574918 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:31.214665076 +0000 UTC m=+245.710554477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zcbwr" (UID: "6f7985a9-98c1-450f-9e35-5401a6574918") : secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:31.222014 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:31.221985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:31.222395 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:31.222029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:31.222395 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:31.222113    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 11:51:31.222395 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:31.222122    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:31.222395 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:31.222159    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm podName:6f7985a9-98c1-450f-9e35-5401a6574918 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:33.22214731 +0000 UTC m=+247.718036712 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zcbwr" (UID: "6f7985a9-98c1-450f-9e35-5401a6574918") : configmap "kube-rbac-proxy" not found
Sep 07 11:51:31.222395 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:31.222178    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls podName:6f7985a9-98c1-450f-9e35-5401a6574918 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:33.222165824 +0000 UTC m=+247.718055226 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zcbwr" (UID: "6f7985a9-98c1-450f-9e35-5401a6574918") : secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:32.202178 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:32.202147    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kwr4n_3c2e42f2-bf8f-4093-b496-af2041e0090e/dns-node-resolver/0.log"
Sep 07 11:51:33.001979 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:33.001952    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9f27p_8722b24c-1b27-4443-aacb-b96b30a04bf3/node-ca/0.log"
Sep 07 11:51:33.237049 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:33.237018    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:33.237211 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:33.237062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:33.237211 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:33.237133    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 11:51:33.237211 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:33.237169    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:33.237211 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:33.237186    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm podName:6f7985a9-98c1-450f-9e35-5401a6574918 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:37.237169895 +0000 UTC m=+251.733059296 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zcbwr" (UID: "6f7985a9-98c1-450f-9e35-5401a6574918") : configmap "kube-rbac-proxy" not found
Sep 07 11:51:33.237370 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:33.237237    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls podName:6f7985a9-98c1-450f-9e35-5401a6574918 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:37.237221641 +0000 UTC m=+251.733111042 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zcbwr" (UID: "6f7985a9-98c1-450f-9e35-5401a6574918") : secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:34.204637 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:34.204603    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vb6cl_02120513-1881-4f03-89fb-c94109d599b4/serve-healthcheck-canary/0.log"
Sep 07 11:51:37.269788 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:37.269753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:37.270224 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:37.269819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:37.270224 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:37.269892    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 11:51:37.270224 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:37.269918    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:37.270224 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:37.269960    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm podName:6f7985a9-98c1-450f-9e35-5401a6574918 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:45.269944342 +0000 UTC m=+259.765833748 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zcbwr" (UID: "6f7985a9-98c1-450f-9e35-5401a6574918") : configmap "kube-rbac-proxy" not found
Sep 07 11:51:37.270224 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:37.269976    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls podName:6f7985a9-98c1-450f-9e35-5401a6574918 nodeName:}" failed. No retries permitted until 2026-09-07 11:51:45.269968984 +0000 UTC m=+259.765858386 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zcbwr" (UID: "6f7985a9-98c1-450f-9e35-5401a6574918") : secret "insights-runtime-extractor-tls" not found
Sep 07 11:51:37.873180 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:37.873143    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs\") pod \"network-metrics-daemon-4dgj8\" (UID: \"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f\") " pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:51:37.873364 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:37.873296    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 07 11:51:37.873364 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:51:37.873357    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs podName:7e4e91b4-23e4-4903-8e2d-cf1d2145b51f nodeName:}" failed. No retries permitted until 2026-09-07 11:53:39.87334265 +0000 UTC m=+374.369232054 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs") pod "network-metrics-daemon-4dgj8" (UID: "7e4e91b4-23e4-4903-8e2d-cf1d2145b51f") : secret "metrics-daemon-secret" not found
Sep 07 11:51:45.330262 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:45.330223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:45.330645 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:45.330312    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:45.330725 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:45.330705    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f7985a9-98c1-450f-9e35-5401a6574918-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:45.332631 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:45.332607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f7985a9-98c1-450f-9e35-5401a6574918-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zcbwr\" (UID: \"6f7985a9-98c1-450f-9e35-5401a6574918\") " pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:45.346070 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:45.346035    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zcbwr"
Sep 07 11:51:45.613568 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:45.613488    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zcbwr"]
Sep 07 11:51:45.616921 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:51:45.616885    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f7985a9_98c1_450f_9e35_5401a6574918.slice/crio-8cf70c0caec80799642f9816cb60cc1378917c630f1c296520246a3186f96cd4 WatchSource:0}: Error finding container 8cf70c0caec80799642f9816cb60cc1378917c630f1c296520246a3186f96cd4: Status 404 returned error can't find the container with id 8cf70c0caec80799642f9816cb60cc1378917c630f1c296520246a3186f96cd4
Sep 07 11:51:45.900277 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:45.900241    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zcbwr" event={"ID":"6f7985a9-98c1-450f-9e35-5401a6574918","Type":"ContainerStarted","Data":"ad548931b0d351a6e24ee8a4405096322c060701fac2c56fc7d87cfec28481d5"}
Sep 07 11:51:45.900441 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:45.900331    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zcbwr" event={"ID":"6f7985a9-98c1-450f-9e35-5401a6574918","Type":"ContainerStarted","Data":"8cf70c0caec80799642f9816cb60cc1378917c630f1c296520246a3186f96cd4"}
Sep 07 11:51:46.905960 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:46.905924    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zcbwr" event={"ID":"6f7985a9-98c1-450f-9e35-5401a6574918","Type":"ContainerStarted","Data":"7f5f55a4c60051b1072a5c4a03e64fed31e08004cd0ed881bf5ae5e1a99a1611"}
Sep 07 11:51:47.910825 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:47.910790    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zcbwr" event={"ID":"6f7985a9-98c1-450f-9e35-5401a6574918","Type":"ContainerStarted","Data":"2e82864758ad2ccbecf5d205351153a78baafeb31ab8897a3054215e36f4c016"}
Sep 07 11:51:47.927917 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:51:47.927868    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zcbwr" podStartSLOduration=17.249578854 podStartE2EDuration="18.927852399s" totalImagesPullingTime="1.678273545s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:51:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:51:47.926373824 +0000 UTC m=+262.422263273" watchObservedRunningTime="2026-09-07 11:51:47.927852399 +0000 UTC m=+262.423741822"
Sep 07 11:52:06.608904 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:52:06.608859    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jsx9s" podUID="17691127-7e46-44e2-a052-edf0873d6f26"
Sep 07 11:52:06.893926 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.893892    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"]
Sep 07 11:52:06.896587 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.896571    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"
Sep 07 11:52:06.899128 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.899102    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 07 11:52:06.899324 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.899185    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 07 11:52:06.899425 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.899400    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 07 11:52:06.900541 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.900518    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 07 11:52:06.900641 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.900601    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 07 11:52:06.900715 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.900659    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-phqq6\""
Sep 07 11:52:06.908561 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.908527    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"]
Sep 07 11:52:06.909328 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.909303    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cfdtb"]
Sep 07 11:52:06.911963 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.911935    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:06.914595 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.914572    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 07 11:52:06.914769 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.914752    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rw24l\""
Sep 07 11:52:06.914838 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.914750    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 07 11:52:06.914884 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.914835    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 07 11:52:06.967592 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.967564    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:52:06.977261 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.977235    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-accelerators-collector-config\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:06.977346 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.977270    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzt4s\" (UniqueName: \"kubernetes.io/projected/aa44cf6c-3099-4514-aa93-28ee27dbf199-kube-api-access-gzt4s\") pod \"openshift-state-metrics-657fd6c54-sfl7k\" (UID: \"aa44cf6c-3099-4514-aa93-28ee27dbf199\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"
Sep 07 11:52:06.977382 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.977341    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-tls\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:06.977416 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.977389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-textfile\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:06.977448 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.977416    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-wtmp\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:06.977448 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.977438    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa44cf6c-3099-4514-aa93-28ee27dbf199-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-sfl7k\" (UID: \"aa44cf6c-3099-4514-aa93-28ee27dbf199\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"
Sep 07 11:52:06.977518 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.977478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:06.977518 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.977515    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa44cf6c-3099-4514-aa93-28ee27dbf199-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-sfl7k\" (UID: \"aa44cf6c-3099-4514-aa93-28ee27dbf199\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"
Sep 07 11:52:06.977581 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.977538    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-root\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:06.977581 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.977566    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-sys\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:06.977642 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.977594    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-metrics-client-ca\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:06.977642 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.977611    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa44cf6c-3099-4514-aa93-28ee27dbf199-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-sfl7k\" (UID: \"aa44cf6c-3099-4514-aa93-28ee27dbf199\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"
Sep 07 11:52:06.977705 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:06.977659    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7djmm\" (UniqueName: \"kubernetes.io/projected/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-kube-api-access-7djmm\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.078654 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.078603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-accelerators-collector-config\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.078654 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.078651    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzt4s\" (UniqueName: \"kubernetes.io/projected/aa44cf6c-3099-4514-aa93-28ee27dbf199-kube-api-access-gzt4s\") pod \"openshift-state-metrics-657fd6c54-sfl7k\" (UID: \"aa44cf6c-3099-4514-aa93-28ee27dbf199\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"
Sep 07 11:52:07.078889 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.078681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-tls\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.078889 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.078721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-textfile\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.078889 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.078743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-wtmp\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.078889 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.078766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa44cf6c-3099-4514-aa93-28ee27dbf199-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-sfl7k\" (UID: \"aa44cf6c-3099-4514-aa93-28ee27dbf199\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"
Sep 07 11:52:07.078889 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.078793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.079128 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:52:07.078894    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 07 11:52:07.079128 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.078912    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-wtmp\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.079128 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.078927    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa44cf6c-3099-4514-aa93-28ee27dbf199-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-sfl7k\" (UID: \"aa44cf6c-3099-4514-aa93-28ee27dbf199\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"
Sep 07 11:52:07.079128 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:52:07.078980    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-tls podName:f2fc51c1-3f3a-4760-93b6-64fa09cedaa3 nodeName:}" failed. No retries permitted until 2026-09-07 11:52:07.578956429 +0000 UTC m=+282.074845847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-tls") pod "node-exporter-cfdtb" (UID: "f2fc51c1-3f3a-4760-93b6-64fa09cedaa3") : secret "node-exporter-tls" not found
Sep 07 11:52:07.079128 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:52:07.078996    2580 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 07 11:52:07.079128 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.079010    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-root\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.079128 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.079042    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-sys\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.079128 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:52:07.079054    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa44cf6c-3099-4514-aa93-28ee27dbf199-openshift-state-metrics-tls podName:aa44cf6c-3099-4514-aa93-28ee27dbf199 nodeName:}" failed. No retries permitted until 2026-09-07 11:52:07.579037668 +0000 UTC m=+282.074927082 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/aa44cf6c-3099-4514-aa93-28ee27dbf199-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-sfl7k" (UID: "aa44cf6c-3099-4514-aa93-28ee27dbf199") : secret "openshift-state-metrics-tls" not found
Sep 07 11:52:07.079128 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.079084    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-sys\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.079128 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.079097    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-metrics-client-ca\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.079128 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.079128    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa44cf6c-3099-4514-aa93-28ee27dbf199-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-sfl7k\" (UID: \"aa44cf6c-3099-4514-aa93-28ee27dbf199\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"
Sep 07 11:52:07.079584 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.079127    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-root\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.079584 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.079184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7djmm\" (UniqueName: \"kubernetes.io/projected/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-kube-api-access-7djmm\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.079584 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.079356    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-accelerators-collector-config\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.079676 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.079630    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-metrics-client-ca\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.079806 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.079789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa44cf6c-3099-4514-aa93-28ee27dbf199-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-sfl7k\" (UID: \"aa44cf6c-3099-4514-aa93-28ee27dbf199\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"
Sep 07 11:52:07.079840 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.079826    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-textfile\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.081117 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.081096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.081351 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.081330    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aa44cf6c-3099-4514-aa93-28ee27dbf199-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-sfl7k\" (UID: \"aa44cf6c-3099-4514-aa93-28ee27dbf199\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"
Sep 07 11:52:07.086963 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.086941    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzt4s\" (UniqueName: \"kubernetes.io/projected/aa44cf6c-3099-4514-aa93-28ee27dbf199-kube-api-access-gzt4s\") pod \"openshift-state-metrics-657fd6c54-sfl7k\" (UID: \"aa44cf6c-3099-4514-aa93-28ee27dbf199\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"
Sep 07 11:52:07.089851 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.089829    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7djmm\" (UniqueName: \"kubernetes.io/projected/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-kube-api-access-7djmm\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.583288 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.583240    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-tls\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.583288 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.583296    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa44cf6c-3099-4514-aa93-28ee27dbf199-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-sfl7k\" (UID: \"aa44cf6c-3099-4514-aa93-28ee27dbf199\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"
Sep 07 11:52:07.585737 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.585699    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2fc51c1-3f3a-4760-93b6-64fa09cedaa3-node-exporter-tls\") pod \"node-exporter-cfdtb\" (UID: \"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3\") " pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.585737 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.585723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa44cf6c-3099-4514-aa93-28ee27dbf199-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-sfl7k\" (UID: \"aa44cf6c-3099-4514-aa93-28ee27dbf199\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"
Sep 07 11:52:07.806076 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.806027    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"
Sep 07 11:52:07.820772 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.820745    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cfdtb"
Sep 07 11:52:07.830692 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:52:07.830653    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2fc51c1_3f3a_4760_93b6_64fa09cedaa3.slice/crio-8ff4ec5c9d7e9c356d00d61fd27c753b08328169d6756b85761775d598f6201f WatchSource:0}: Error finding container 8ff4ec5c9d7e9c356d00d61fd27c753b08328169d6756b85761775d598f6201f: Status 404 returned error can't find the container with id 8ff4ec5c9d7e9c356d00d61fd27c753b08328169d6756b85761775d598f6201f
Sep 07 11:52:07.972290 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:07.972254    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cfdtb" event={"ID":"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3","Type":"ContainerStarted","Data":"8ff4ec5c9d7e9c356d00d61fd27c753b08328169d6756b85761775d598f6201f"}
Sep 07 11:52:08.076611 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:08.076578    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k"]
Sep 07 11:52:08.079446 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:52:08.079421    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa44cf6c_3099_4514_aa93_28ee27dbf199.slice/crio-35221b9a8bc389207f19b780a58d41cef724a261a254159cb22e00ca75776e54 WatchSource:0}: Error finding container 35221b9a8bc389207f19b780a58d41cef724a261a254159cb22e00ca75776e54: Status 404 returned error can't find the container with id 35221b9a8bc389207f19b780a58d41cef724a261a254159cb22e00ca75776e54
Sep 07 11:52:08.980803 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:08.980770    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k" event={"ID":"aa44cf6c-3099-4514-aa93-28ee27dbf199","Type":"ContainerStarted","Data":"37b15d4c768dd8a733e97fd62c5c6a08b09b798b450cb35d5b806b3a5033fe62"}
Sep 07 11:52:08.981292 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:08.980838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k" event={"ID":"aa44cf6c-3099-4514-aa93-28ee27dbf199","Type":"ContainerStarted","Data":"8017aa5a435db6fe28194d82d0bf2f3aa96b06ae71e450ebebc091cc4d4052f1"}
Sep 07 11:52:08.981292 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:08.980868    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k" event={"ID":"aa44cf6c-3099-4514-aa93-28ee27dbf199","Type":"ContainerStarted","Data":"35221b9a8bc389207f19b780a58d41cef724a261a254159cb22e00ca75776e54"}
Sep 07 11:52:09.985993 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:09.985952    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k" event={"ID":"aa44cf6c-3099-4514-aa93-28ee27dbf199","Type":"ContainerStarted","Data":"61aa563d040d0f3c8a1061463f2416f8b8ea57b891ca45be428a68501baed952"}
Sep 07 11:52:09.987609 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:09.987580    2580 generic.go:356] "Generic (PLEG): container finished" podID="f2fc51c1-3f3a-4760-93b6-64fa09cedaa3" containerID="539c72f6cc2a48c7bf3327288f4cffb379bb1b2f2ed7f7aad1da535f28e6c3b4" exitCode=0
Sep 07 11:52:09.987702 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:09.987617    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cfdtb" event={"ID":"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3","Type":"ContainerDied","Data":"539c72f6cc2a48c7bf3327288f4cffb379bb1b2f2ed7f7aad1da535f28e6c3b4"}
Sep 07 11:52:10.005128 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:10.005080    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-sfl7k" podStartSLOduration=2.776097707 podStartE2EDuration="4.005058916s" totalImagesPullingTime="1.228961209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:52:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:52:10.003777488 +0000 UTC m=+284.499666915" watchObservedRunningTime="2026-09-07 11:52:10.005058916 +0000 UTC m=+284.500948340"
Sep 07 11:52:10.305541 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:10.305445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:52:10.307888 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:10.307865    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/17691127-7e46-44e2-a052-edf0873d6f26-metrics-tls\") pod \"dns-default-jsx9s\" (UID: \"17691127-7e46-44e2-a052-edf0873d6f26\") " pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:52:10.571034 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:10.570953    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kd29r\""
Sep 07 11:52:10.578887 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:10.578869    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:52:10.845537 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:10.845448    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jsx9s"]
Sep 07 11:52:10.848668 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:52:10.848634    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17691127_7e46_44e2_a052_edf0873d6f26.slice/crio-a64b13758c6ef28c603e342a097846e1ff176db43b843b7ebc488e75be9c7be1 WatchSource:0}: Error finding container a64b13758c6ef28c603e342a097846e1ff176db43b843b7ebc488e75be9c7be1: Status 404 returned error can't find the container with id a64b13758c6ef28c603e342a097846e1ff176db43b843b7ebc488e75be9c7be1
Sep 07 11:52:10.993045 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:10.993010    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cfdtb" event={"ID":"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3","Type":"ContainerStarted","Data":"45c11e99f9d487e4ec2e084bd0bc72da548aacc7e562bd5ffc55f50e45f2edec"}
Sep 07 11:52:10.993505 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:10.993061    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cfdtb" event={"ID":"f2fc51c1-3f3a-4760-93b6-64fa09cedaa3","Type":"ContainerStarted","Data":"7746d3d8b482234f3a03ca66ef60812631e0526bee4ce3b43660525f22e4d520"}
Sep 07 11:52:10.994249 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:10.994224    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jsx9s" event={"ID":"17691127-7e46-44e2-a052-edf0873d6f26","Type":"ContainerStarted","Data":"a64b13758c6ef28c603e342a097846e1ff176db43b843b7ebc488e75be9c7be1"}
Sep 07 11:52:11.011477 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:11.011433    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cfdtb" podStartSLOduration=3.651657364 podStartE2EDuration="5.0114179s" totalImagesPullingTime="1.291681025s" totalInitContainerRuntime="68.079511ms" isStatefulPod=true podCreationTimestamp="2026-09-07 11:52:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:52:11.010190724 +0000 UTC m=+285.506080147" watchObservedRunningTime="2026-09-07 11:52:11.0114179 +0000 UTC m=+285.507307323"
Sep 07 11:52:13.007368 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.007330    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jsx9s" event={"ID":"17691127-7e46-44e2-a052-edf0873d6f26","Type":"ContainerStarted","Data":"72c247ab22e0e4245a0850564e80030ad3d4bca3020a411c5021f5193ed94b57"}
Sep 07 11:52:13.007772 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.007406    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jsx9s" event={"ID":"17691127-7e46-44e2-a052-edf0873d6f26","Type":"ContainerStarted","Data":"5a0898462a34db6b701cc1f807cf9ea9c91ebd2186a5b12ad01cba6c5690e361"}
Sep 07 11:52:13.007772 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.007469    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:52:13.024264 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.024218    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jsx9s" podStartSLOduration=251.566909982 podStartE2EDuration="4m13.024181668s" totalImagesPullingTime="1.457271686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:52:13.023841846 +0000 UTC m=+287.519731270" watchObservedRunningTime="2026-09-07 11:52:13.024181668 +0000 UTC m=+287.520071091"
Sep 07 11:52:13.128443 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.128407    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 11:52:13.131916 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.131893    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.134589 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.134566    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 07 11:52:13.134704 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.134575    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 07 11:52:13.135060 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.135006    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 07 11:52:13.135060 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.135032    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 07 11:52:13.135266 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.135060    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 07 11:52:13.135266 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.135066    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 07 11:52:13.135266 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.135138    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 07 11:52:13.135266 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.135209    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 07 11:52:13.135529 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.135398    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 07 11:52:13.135592 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.135537    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 07 11:52:13.135592 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.135588    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 07 11:52:13.135709 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.135652    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 07 11:52:13.135771 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.135755    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xdg5b\""
Sep 07 11:52:13.135855 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.135838    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7k7hqgdd3o3tq\""
Sep 07 11:52:13.135911 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.135852    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 07 11:52:13.135960 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.135937    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 07 11:52:13.139119 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.139099    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 07 11:52:13.151377 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.151350    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 11:52:13.226252 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226252 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226254    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-config\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226448 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-web-config\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226448 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226303    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226448 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226377    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226448 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m4fl\" (UniqueName: \"kubernetes.io/projected/e3612b76-503b-42e9-895d-87969c738012-kube-api-access-7m4fl\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226448 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226425    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e3612b76-503b-42e9-895d-87969c738012-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226448 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226443    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226649 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226649 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226539    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226649 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226564    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226649 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226590    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226649 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226649 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226633    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226827 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e3612b76-503b-42e9-895d-87969c738012-config-out\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226827 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226682    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226827 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226700    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226827 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226715    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226827 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226779    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.226827 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.226800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.327617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.327534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.327617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.327571    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-config\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.327617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.327593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-web-config\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.327617 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.327615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.327969 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.327853    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.327969 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.327875    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m4fl\" (UniqueName: \"kubernetes.io/projected/e3612b76-503b-42e9-895d-87969c738012-kube-api-access-7m4fl\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.327969 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.327892    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e3612b76-503b-42e9-895d-87969c738012-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.327969 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.327907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.327969 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.327922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.327969 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.327951    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.328350 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.327980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.328350 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.328003    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.328350 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.328028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.328350 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.328064    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.328350 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.328093    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e3612b76-503b-42e9-895d-87969c738012-config-out\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.328350 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.328115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.328350 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.328150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.328350 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.328179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.328350 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.328255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.328350 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.328286    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.329114 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.329086    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.330412 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.329889    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.331374 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.330805    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.331374 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.330934    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.331374 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.331015    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-config\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.331374 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.331337    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.331641 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.331479    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.331641 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.331596    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.331756 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.331635    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-web-config\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.331756 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.331637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e3612b76-503b-42e9-895d-87969c738012-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.331965 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.331939    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.332030 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.331951    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.332427 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.332405    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.332683 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.332660    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.332759 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.332707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.333619 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.333593    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.333849 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.333832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e3612b76-503b-42e9-895d-87969c738012-config-out\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.334086 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.334066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.334271 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.334254    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.340424 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.340399    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m4fl\" (UniqueName: \"kubernetes.io/projected/e3612b76-503b-42e9-895d-87969c738012-kube-api-access-7m4fl\") pod \"prometheus-k8s-0\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.441385 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.441347    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:13.729821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:13.729691    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 11:52:13.732265 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:52:13.732235    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3612b76_503b_42e9_895d_87969c738012.slice/crio-9f1505b58f6bba26dbdb7a7c573dc2235403aad10f9377fa3d1388ae17d7a9b0 WatchSource:0}: Error finding container 9f1505b58f6bba26dbdb7a7c573dc2235403aad10f9377fa3d1388ae17d7a9b0: Status 404 returned error can't find the container with id 9f1505b58f6bba26dbdb7a7c573dc2235403aad10f9377fa3d1388ae17d7a9b0
Sep 07 11:52:14.011162 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:14.011063    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3612b76-503b-42e9-895d-87969c738012","Type":"ContainerStarted","Data":"9f1505b58f6bba26dbdb7a7c573dc2235403aad10f9377fa3d1388ae17d7a9b0"}
Sep 07 11:52:15.015493 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:15.015458    2580 generic.go:356] "Generic (PLEG): container finished" podID="e3612b76-503b-42e9-895d-87969c738012" containerID="7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f" exitCode=0
Sep 07 11:52:15.015867 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:15.015525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3612b76-503b-42e9-895d-87969c738012","Type":"ContainerDied","Data":"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f"}
Sep 07 11:52:18.137896 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:18.137851    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h" podUID="dce4a004-6319-43a2-ac7e-ed7aafc22569" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 07 11:52:19.032605 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:19.032565    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3612b76-503b-42e9-895d-87969c738012","Type":"ContainerStarted","Data":"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec"}
Sep 07 11:52:19.032756 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:19.032639    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3612b76-503b-42e9-895d-87969c738012","Type":"ContainerStarted","Data":"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac"}
Sep 07 11:52:21.042959 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:21.042924    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3612b76-503b-42e9-895d-87969c738012","Type":"ContainerStarted","Data":"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2"}
Sep 07 11:52:21.043347 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:21.042988    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3612b76-503b-42e9-895d-87969c738012","Type":"ContainerStarted","Data":"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784"}
Sep 07 11:52:21.043347 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:21.043013    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3612b76-503b-42e9-895d-87969c738012","Type":"ContainerStarted","Data":"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f"}
Sep 07 11:52:21.043347 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:21.043036    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3612b76-503b-42e9-895d-87969c738012","Type":"ContainerStarted","Data":"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234"}
Sep 07 11:52:21.071698 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:21.071642    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.341025888 podStartE2EDuration="8.071620962s" totalImagesPullingTime="5.708303519s" totalInitContainerRuntime="22.291555ms" isStatefulPod=false podCreationTimestamp="2026-09-07 11:52:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:52:21.069817616 +0000 UTC m=+295.565707064" watchObservedRunningTime="2026-09-07 11:52:21.071620962 +0000 UTC m=+295.567510393"
Sep 07 11:52:23.005649 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:52:23.005596    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-4dgj8" podUID="7e4e91b4-23e4-4903-8e2d-cf1d2145b51f"
Sep 07 11:52:23.013625 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:23.013598    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jsx9s"
Sep 07 11:52:23.441500 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:23.441455    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:52:25.909822 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:25.909795    2580 kubelet.go:1648] "Image garbage collection succeeded"
Sep 07 11:52:28.137351 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:28.137315    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h" podUID="dce4a004-6319-43a2-ac7e-ed7aafc22569" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 07 11:52:38.004927 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:38.004886    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:52:38.136924 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:38.136888    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h" podUID="dce4a004-6319-43a2-ac7e-ed7aafc22569" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 07 11:52:38.137078 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:38.136953    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h"
Sep 07 11:52:38.137423 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:38.137394    2580 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"3a202c85c6d4210cfc2c9a1515aa6ef37f43d023b88644df3e8b6fcd4128744a"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 07 11:52:38.137476 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:38.137444    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h" podUID="dce4a004-6319-43a2-ac7e-ed7aafc22569" containerName="service-proxy" containerID="cri-o://3a202c85c6d4210cfc2c9a1515aa6ef37f43d023b88644df3e8b6fcd4128744a" gracePeriod=30
Sep 07 11:52:39.108258 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:39.108217    2580 generic.go:356] "Generic (PLEG): container finished" podID="dce4a004-6319-43a2-ac7e-ed7aafc22569" containerID="3a202c85c6d4210cfc2c9a1515aa6ef37f43d023b88644df3e8b6fcd4128744a" exitCode=2
Sep 07 11:52:39.108258 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:39.108263    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h" event={"ID":"dce4a004-6319-43a2-ac7e-ed7aafc22569","Type":"ContainerDied","Data":"3a202c85c6d4210cfc2c9a1515aa6ef37f43d023b88644df3e8b6fcd4128744a"}
Sep 07 11:52:39.108680 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:52:39.108295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-68d67bfb57-d9d9h" event={"ID":"dce4a004-6319-43a2-ac7e-ed7aafc22569","Type":"ContainerStarted","Data":"a89e572e10f968691f3b8f399df199c2c16e4af9494a8383a37794498ad25ad6"}
Sep 07 11:53:13.442264 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:13.442190    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:13.461304 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:13.461276    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:14.243308 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:14.243275    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:31.493462 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.493417    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 11:53:31.493968 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.493786    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="prometheus" containerID="cri-o://bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac" gracePeriod=600
Sep 07 11:53:31.493968 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.493814    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="kube-rbac-proxy" containerID="cri-o://68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784" gracePeriod=600
Sep 07 11:53:31.493968 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.493832    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="thanos-sidecar" containerID="cri-o://b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234" gracePeriod=600
Sep 07 11:53:31.493968 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.493868    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="config-reloader" containerID="cri-o://183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec" gracePeriod=600
Sep 07 11:53:31.493968 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.493877    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="kube-rbac-proxy-thanos" containerID="cri-o://175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2" gracePeriod=600
Sep 07 11:53:31.493968 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.493840    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="kube-rbac-proxy-web" containerID="cri-o://80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f" gracePeriod=600
Sep 07 11:53:31.728119 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.728095    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:31.865488 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865397    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-kubelet-serving-ca-bundle\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.865488 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865440    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-web-config\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.865488 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865464    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e3612b76-503b-42e9-895d-87969c738012-tls-assets\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.865488 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865483    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7m4fl\" (UniqueName: \"kubernetes.io/projected/e3612b76-503b-42e9-895d-87969c738012-kube-api-access-7m4fl\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.865821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865504    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-serving-certs-ca-bundle\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.865821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865531    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.865821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865556    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-thanos-prometheus-http-client-file\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.865821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865599    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-trusted-ca-bundle\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.865821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865624    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-kube-rbac-proxy\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.865821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865655    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-grpc-tls\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.865821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865679    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-1\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.865821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865708    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e3612b76-503b-42e9-895d-87969c738012-config-out\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.865821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865752    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-metrics-client-ca\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.865821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865782    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.865821 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865809    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-2\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.866360 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865831    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:53:31.866360 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865859    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-db\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.866360 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865887    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-metrics-client-certs\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.866360 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865922    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-0\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.866360 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865946    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-tls\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.866360 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865972    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-config\") pod \"e3612b76-503b-42e9-895d-87969c738012\" (UID: \"e3612b76-503b-42e9-895d-87969c738012\") "
Sep 07 11:53:31.866360 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.866240    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.869186 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.865937    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:53:31.869186 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.866286    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:53:31.869186 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.866653    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:53:31.869186 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.867845    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:53:31.869186 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.868263    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:53:31.869186 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.868333    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3612b76-503b-42e9-895d-87969c738012-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:53:31.869186 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.868766    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:53:31.869674 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.869273    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:53:31.869674 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.869262    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:31.869674 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.869441    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3612b76-503b-42e9-895d-87969c738012-kube-api-access-7m4fl" (OuterVolumeSpecName: "kube-api-access-7m4fl") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "kube-api-access-7m4fl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:53:31.869674 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.869474    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:31.869882 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.869702    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:31.870043 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.869995    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:31.870255 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.870166    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:31.870763 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.870737    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:31.870888 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.870874    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-config" (OuterVolumeSpecName: "config") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:31.871133 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.871104    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3612b76-503b-42e9-895d-87969c738012-config-out" (OuterVolumeSpecName: "config-out") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:53:31.871278 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.871253    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:31.878062 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.878038    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-web-config" (OuterVolumeSpecName: "web-config") pod "e3612b76-503b-42e9-895d-87969c738012" (UID: "e3612b76-503b-42e9-895d-87969c738012"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:53:31.967564 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967520    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-db\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967564 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967559    2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-metrics-client-certs\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967564 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967569    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967564 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967578    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967903 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967587    2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-config\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967903 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967596    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-web-config\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967903 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967604    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e3612b76-503b-42e9-895d-87969c738012-tls-assets\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967903 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967612    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7m4fl\" (UniqueName: \"kubernetes.io/projected/e3612b76-503b-42e9-895d-87969c738012-kube-api-access-7m4fl\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967903 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967619    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967903 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967628    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967903 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967637    2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967903 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967645    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967903 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967653    2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-kube-rbac-proxy\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967903 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967661    2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-grpc-tls\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967903 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967668    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967903 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967676    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e3612b76-503b-42e9-895d-87969c738012-config-out\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967903 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967684    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-configmap-metrics-client-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967903 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967691    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3612b76-503b-42e9-895d-87969c738012-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:31.967903 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:31.967703    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e3612b76-503b-42e9-895d-87969c738012-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:53:32.289975 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.289946    2580 generic.go:356] "Generic (PLEG): container finished" podID="e3612b76-503b-42e9-895d-87969c738012" containerID="175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2" exitCode=0
Sep 07 11:53:32.289975 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.289969    2580 generic.go:356] "Generic (PLEG): container finished" podID="e3612b76-503b-42e9-895d-87969c738012" containerID="68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784" exitCode=0
Sep 07 11:53:32.289975 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.289978    2580 generic.go:356] "Generic (PLEG): container finished" podID="e3612b76-503b-42e9-895d-87969c738012" containerID="80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f" exitCode=0
Sep 07 11:53:32.290210 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.289987    2580 generic.go:356] "Generic (PLEG): container finished" podID="e3612b76-503b-42e9-895d-87969c738012" containerID="b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234" exitCode=0
Sep 07 11:53:32.290210 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.289996    2580 generic.go:356] "Generic (PLEG): container finished" podID="e3612b76-503b-42e9-895d-87969c738012" containerID="183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec" exitCode=0
Sep 07 11:53:32.290210 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.290003    2580 generic.go:356] "Generic (PLEG): container finished" podID="e3612b76-503b-42e9-895d-87969c738012" containerID="bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac" exitCode=0
Sep 07 11:53:32.290210 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.290005    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3612b76-503b-42e9-895d-87969c738012","Type":"ContainerDied","Data":"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2"}
Sep 07 11:53:32.290210 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.290017    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.290210 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.290077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3612b76-503b-42e9-895d-87969c738012","Type":"ContainerDied","Data":"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784"}
Sep 07 11:53:32.290210 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.290106    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3612b76-503b-42e9-895d-87969c738012","Type":"ContainerDied","Data":"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f"}
Sep 07 11:53:32.290210 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.290134    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3612b76-503b-42e9-895d-87969c738012","Type":"ContainerDied","Data":"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234"}
Sep 07 11:53:32.290210 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.290145    2580 scope.go:117] "RemoveContainer" containerID="175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2"
Sep 07 11:53:32.290210 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.290174    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3612b76-503b-42e9-895d-87969c738012","Type":"ContainerDied","Data":"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec"}
Sep 07 11:53:32.290594 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.290231    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3612b76-503b-42e9-895d-87969c738012","Type":"ContainerDied","Data":"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac"}
Sep 07 11:53:32.290594 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.290271    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e3612b76-503b-42e9-895d-87969c738012","Type":"ContainerDied","Data":"9f1505b58f6bba26dbdb7a7c573dc2235403aad10f9377fa3d1388ae17d7a9b0"}
Sep 07 11:53:32.298151 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.298133    2580 scope.go:117] "RemoveContainer" containerID="68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784"
Sep 07 11:53:32.304788 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.304766    2580 scope.go:117] "RemoveContainer" containerID="80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f"
Sep 07 11:53:32.311464 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.311444    2580 scope.go:117] "RemoveContainer" containerID="b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234"
Sep 07 11:53:32.312192 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.312171    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 11:53:32.316464 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.316437    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 11:53:32.318295 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.318276    2580 scope.go:117] "RemoveContainer" containerID="183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec"
Sep 07 11:53:32.324547 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.324526    2580 scope.go:117] "RemoveContainer" containerID="bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac"
Sep 07 11:53:32.331125 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.331110    2580 scope.go:117] "RemoveContainer" containerID="7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f"
Sep 07 11:53:32.337291 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.337281    2580 scope.go:117] "RemoveContainer" containerID="175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2"
Sep 07 11:53:32.337563 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:53:32.337535    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2\": container with ID starting with 175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2 not found: ID does not exist" containerID="175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2"
Sep 07 11:53:32.337612 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.337570    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2"} err="failed to get container status \"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2\": rpc error: code = NotFound desc = could not find container \"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2\": container with ID starting with 175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2 not found: ID does not exist"
Sep 07 11:53:32.337612 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.337587    2580 scope.go:117] "RemoveContainer" containerID="68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784"
Sep 07 11:53:32.337851 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:53:32.337834    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784\": container with ID starting with 68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784 not found: ID does not exist" containerID="68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784"
Sep 07 11:53:32.337895 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.337858    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784"} err="failed to get container status \"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784\": rpc error: code = NotFound desc = could not find container \"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784\": container with ID starting with 68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784 not found: ID does not exist"
Sep 07 11:53:32.337895 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.337874    2580 scope.go:117] "RemoveContainer" containerID="80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f"
Sep 07 11:53:32.338112 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:53:32.338095    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f\": container with ID starting with 80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f not found: ID does not exist" containerID="80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f"
Sep 07 11:53:32.338151 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.338117    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f"} err="failed to get container status \"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f\": rpc error: code = NotFound desc = could not find container \"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f\": container with ID starting with 80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f not found: ID does not exist"
Sep 07 11:53:32.338151 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.338133    2580 scope.go:117] "RemoveContainer" containerID="b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234"
Sep 07 11:53:32.338390 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:53:32.338373    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234\": container with ID starting with b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234 not found: ID does not exist" containerID="b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234"
Sep 07 11:53:32.338437 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.338394    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234"} err="failed to get container status \"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234\": rpc error: code = NotFound desc = could not find container \"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234\": container with ID starting with b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234 not found: ID does not exist"
Sep 07 11:53:32.338437 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.338408    2580 scope.go:117] "RemoveContainer" containerID="183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec"
Sep 07 11:53:32.338653 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:53:32.338635    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec\": container with ID starting with 183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec not found: ID does not exist" containerID="183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec"
Sep 07 11:53:32.338695 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.338658    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec"} err="failed to get container status \"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec\": rpc error: code = NotFound desc = could not find container \"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec\": container with ID starting with 183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec not found: ID does not exist"
Sep 07 11:53:32.338695 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.338672    2580 scope.go:117] "RemoveContainer" containerID="bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac"
Sep 07 11:53:32.338890 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:53:32.338876    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac\": container with ID starting with bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac not found: ID does not exist" containerID="bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac"
Sep 07 11:53:32.338933 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.338895    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac"} err="failed to get container status \"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac\": rpc error: code = NotFound desc = could not find container \"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac\": container with ID starting with bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac not found: ID does not exist"
Sep 07 11:53:32.338933 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.338909    2580 scope.go:117] "RemoveContainer" containerID="7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f"
Sep 07 11:53:32.339129 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:53:32.339115    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f\": container with ID starting with 7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f not found: ID does not exist" containerID="7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f"
Sep 07 11:53:32.339171 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.339131    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f"} err="failed to get container status \"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f\": rpc error: code = NotFound desc = could not find container \"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f\": container with ID starting with 7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f not found: ID does not exist"
Sep 07 11:53:32.339171 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.339144    2580 scope.go:117] "RemoveContainer" containerID="175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2"
Sep 07 11:53:32.339375 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.339358    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2"} err="failed to get container status \"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2\": rpc error: code = NotFound desc = could not find container \"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2\": container with ID starting with 175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2 not found: ID does not exist"
Sep 07 11:53:32.339417 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.339376    2580 scope.go:117] "RemoveContainer" containerID="68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784"
Sep 07 11:53:32.339581 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.339566    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784"} err="failed to get container status \"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784\": rpc error: code = NotFound desc = could not find container \"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784\": container with ID starting with 68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784 not found: ID does not exist"
Sep 07 11:53:32.339626 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.339582    2580 scope.go:117] "RemoveContainer" containerID="80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f"
Sep 07 11:53:32.339774 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.339756    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f"} err="failed to get container status \"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f\": rpc error: code = NotFound desc = could not find container \"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f\": container with ID starting with 80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f not found: ID does not exist"
Sep 07 11:53:32.339814 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.339776    2580 scope.go:117] "RemoveContainer" containerID="b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234"
Sep 07 11:53:32.339959 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.339944    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234"} err="failed to get container status \"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234\": rpc error: code = NotFound desc = could not find container \"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234\": container with ID starting with b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234 not found: ID does not exist"
Sep 07 11:53:32.340001 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.339959    2580 scope.go:117] "RemoveContainer" containerID="183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec"
Sep 07 11:53:32.340142 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.340127    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec"} err="failed to get container status \"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec\": rpc error: code = NotFound desc = could not find container \"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec\": container with ID starting with 183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec not found: ID does not exist"
Sep 07 11:53:32.340185 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.340141    2580 scope.go:117] "RemoveContainer" containerID="bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac"
Sep 07 11:53:32.340353 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.340337    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac"} err="failed to get container status \"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac\": rpc error: code = NotFound desc = could not find container \"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac\": container with ID starting with bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac not found: ID does not exist"
Sep 07 11:53:32.340399 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.340354    2580 scope.go:117] "RemoveContainer" containerID="7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f"
Sep 07 11:53:32.340549 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.340535    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f"} err="failed to get container status \"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f\": rpc error: code = NotFound desc = could not find container \"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f\": container with ID starting with 7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f not found: ID does not exist"
Sep 07 11:53:32.340549 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.340548    2580 scope.go:117] "RemoveContainer" containerID="175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2"
Sep 07 11:53:32.340776 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.340757    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2"} err="failed to get container status \"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2\": rpc error: code = NotFound desc = could not find container \"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2\": container with ID starting with 175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2 not found: ID does not exist"
Sep 07 11:53:32.340842 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.340779    2580 scope.go:117] "RemoveContainer" containerID="68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784"
Sep 07 11:53:32.340986 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.340971    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784"} err="failed to get container status \"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784\": rpc error: code = NotFound desc = could not find container \"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784\": container with ID starting with 68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784 not found: ID does not exist"
Sep 07 11:53:32.341030 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.340986    2580 scope.go:117] "RemoveContainer" containerID="80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f"
Sep 07 11:53:32.341176 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.341159    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f"} err="failed to get container status \"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f\": rpc error: code = NotFound desc = could not find container \"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f\": container with ID starting with 80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f not found: ID does not exist"
Sep 07 11:53:32.341243 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.341176    2580 scope.go:117] "RemoveContainer" containerID="b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234"
Sep 07 11:53:32.341378 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.341361    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234"} err="failed to get container status \"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234\": rpc error: code = NotFound desc = could not find container \"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234\": container with ID starting with b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234 not found: ID does not exist"
Sep 07 11:53:32.341455 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.341380    2580 scope.go:117] "RemoveContainer" containerID="183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec"
Sep 07 11:53:32.341614 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.341572    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec"} err="failed to get container status \"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec\": rpc error: code = NotFound desc = could not find container \"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec\": container with ID starting with 183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec not found: ID does not exist"
Sep 07 11:53:32.341614 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.341590    2580 scope.go:117] "RemoveContainer" containerID="bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac"
Sep 07 11:53:32.341801 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.341775    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac"} err="failed to get container status \"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac\": rpc error: code = NotFound desc = could not find container \"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac\": container with ID starting with bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac not found: ID does not exist"
Sep 07 11:53:32.341801 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.341801    2580 scope.go:117] "RemoveContainer" containerID="7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f"
Sep 07 11:53:32.342106 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.342084    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f"} err="failed to get container status \"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f\": rpc error: code = NotFound desc = could not find container \"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f\": container with ID starting with 7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f not found: ID does not exist"
Sep 07 11:53:32.342175 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.342114    2580 scope.go:117] "RemoveContainer" containerID="175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2"
Sep 07 11:53:32.342403 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.342386    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2"} err="failed to get container status \"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2\": rpc error: code = NotFound desc = could not find container \"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2\": container with ID starting with 175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2 not found: ID does not exist"
Sep 07 11:53:32.342458 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.342403    2580 scope.go:117] "RemoveContainer" containerID="68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784"
Sep 07 11:53:32.342695 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.342668    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784"} err="failed to get container status \"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784\": rpc error: code = NotFound desc = could not find container \"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784\": container with ID starting with 68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784 not found: ID does not exist"
Sep 07 11:53:32.342825 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.342695    2580 scope.go:117] "RemoveContainer" containerID="80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f"
Sep 07 11:53:32.342991 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.342967    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f"} err="failed to get container status \"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f\": rpc error: code = NotFound desc = could not find container \"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f\": container with ID starting with 80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f not found: ID does not exist"
Sep 07 11:53:32.343080 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.342988    2580 scope.go:117] "RemoveContainer" containerID="b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234"
Sep 07 11:53:32.343287 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343269    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234"} err="failed to get container status \"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234\": rpc error: code = NotFound desc = could not find container \"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234\": container with ID starting with b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234 not found: ID does not exist"
Sep 07 11:53:32.343332 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343295    2580 scope.go:117] "RemoveContainer" containerID="183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec"
Sep 07 11:53:32.343364 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343274    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 11:53:32.343551 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343532    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec"} err="failed to get container status \"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec\": rpc error: code = NotFound desc = could not find container \"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec\": container with ID starting with 183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec not found: ID does not exist"
Sep 07 11:53:32.343596 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343552    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="kube-rbac-proxy-web"
Sep 07 11:53:32.343596 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343568    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="kube-rbac-proxy-web"
Sep 07 11:53:32.343596 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343579    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="config-reloader"
Sep 07 11:53:32.343596 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343585    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="config-reloader"
Sep 07 11:53:32.343596 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343590    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="kube-rbac-proxy"
Sep 07 11:53:32.343596 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343597    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="kube-rbac-proxy"
Sep 07 11:53:32.343766 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343604    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="kube-rbac-proxy-thanos"
Sep 07 11:53:32.343766 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343613    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="kube-rbac-proxy-thanos"
Sep 07 11:53:32.343766 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343626    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="prometheus"
Sep 07 11:53:32.343766 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343632    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="prometheus"
Sep 07 11:53:32.343766 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343638    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="thanos-sidecar"
Sep 07 11:53:32.343766 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343642    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="thanos-sidecar"
Sep 07 11:53:32.343766 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343649    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="init-config-reloader"
Sep 07 11:53:32.343766 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343654    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="init-config-reloader"
Sep 07 11:53:32.343766 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343693    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="thanos-sidecar"
Sep 07 11:53:32.343766 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343704    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="kube-rbac-proxy-web"
Sep 07 11:53:32.343766 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343710    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="config-reloader"
Sep 07 11:53:32.343766 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343716    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="kube-rbac-proxy"
Sep 07 11:53:32.343766 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343722    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="prometheus"
Sep 07 11:53:32.343766 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343727    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3612b76-503b-42e9-895d-87969c738012" containerName="kube-rbac-proxy-thanos"
Sep 07 11:53:32.343766 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343553    2580 scope.go:117] "RemoveContainer" containerID="bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac"
Sep 07 11:53:32.344275 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343963    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac"} err="failed to get container status \"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac\": rpc error: code = NotFound desc = could not find container \"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac\": container with ID starting with bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac not found: ID does not exist"
Sep 07 11:53:32.344275 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.343975    2580 scope.go:117] "RemoveContainer" containerID="7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f"
Sep 07 11:53:32.344275 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.344156    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f"} err="failed to get container status \"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f\": rpc error: code = NotFound desc = could not find container \"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f\": container with ID starting with 7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f not found: ID does not exist"
Sep 07 11:53:32.344275 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.344168    2580 scope.go:117] "RemoveContainer" containerID="175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2"
Sep 07 11:53:32.344422 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.344399    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2"} err="failed to get container status \"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2\": rpc error: code = NotFound desc = could not find container \"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2\": container with ID starting with 175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2 not found: ID does not exist"
Sep 07 11:53:32.344422 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.344412    2580 scope.go:117] "RemoveContainer" containerID="68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784"
Sep 07 11:53:32.344655 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.344632    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784"} err="failed to get container status \"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784\": rpc error: code = NotFound desc = could not find container \"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784\": container with ID starting with 68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784 not found: ID does not exist"
Sep 07 11:53:32.344704 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.344657    2580 scope.go:117] "RemoveContainer" containerID="80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f"
Sep 07 11:53:32.344877 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.344861    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f"} err="failed to get container status \"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f\": rpc error: code = NotFound desc = could not find container \"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f\": container with ID starting with 80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f not found: ID does not exist"
Sep 07 11:53:32.344877 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.344877    2580 scope.go:117] "RemoveContainer" containerID="b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234"
Sep 07 11:53:32.345100 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.345083    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234"} err="failed to get container status \"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234\": rpc error: code = NotFound desc = could not find container \"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234\": container with ID starting with b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234 not found: ID does not exist"
Sep 07 11:53:32.345100 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.345100    2580 scope.go:117] "RemoveContainer" containerID="183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec"
Sep 07 11:53:32.345275 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.345260    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec"} err="failed to get container status \"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec\": rpc error: code = NotFound desc = could not find container \"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec\": container with ID starting with 183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec not found: ID does not exist"
Sep 07 11:53:32.345275 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.345275    2580 scope.go:117] "RemoveContainer" containerID="bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac"
Sep 07 11:53:32.345508 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.345486    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac"} err="failed to get container status \"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac\": rpc error: code = NotFound desc = could not find container \"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac\": container with ID starting with bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac not found: ID does not exist"
Sep 07 11:53:32.345508 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.345507    2580 scope.go:117] "RemoveContainer" containerID="7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f"
Sep 07 11:53:32.345756 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.345740    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f"} err="failed to get container status \"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f\": rpc error: code = NotFound desc = could not find container \"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f\": container with ID starting with 7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f not found: ID does not exist"
Sep 07 11:53:32.345809 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.345756    2580 scope.go:117] "RemoveContainer" containerID="175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2"
Sep 07 11:53:32.345974 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.345959    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2"} err="failed to get container status \"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2\": rpc error: code = NotFound desc = could not find container \"175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2\": container with ID starting with 175b760565024720ae301e10efb964ba58185c582a7dfb127781b27926117ce2 not found: ID does not exist"
Sep 07 11:53:32.345974 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.345973    2580 scope.go:117] "RemoveContainer" containerID="68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784"
Sep 07 11:53:32.346187 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.346171    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784"} err="failed to get container status \"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784\": rpc error: code = NotFound desc = could not find container \"68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784\": container with ID starting with 68b769b02580ee709268a1b1f385cac82c39e5d07d5588859ac6977702adc784 not found: ID does not exist"
Sep 07 11:53:32.346249 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.346188    2580 scope.go:117] "RemoveContainer" containerID="80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f"
Sep 07 11:53:32.346423 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.346401    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f"} err="failed to get container status \"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f\": rpc error: code = NotFound desc = could not find container \"80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f\": container with ID starting with 80139cb3e4330c6b4c5fe31527ca61ecd9bf934b2a872d2dfaf36f38decd500f not found: ID does not exist"
Sep 07 11:53:32.346467 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.346423    2580 scope.go:117] "RemoveContainer" containerID="b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234"
Sep 07 11:53:32.346623 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.346608    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234"} err="failed to get container status \"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234\": rpc error: code = NotFound desc = could not find container \"b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234\": container with ID starting with b8fdfb9c0adb731174a1406681731ef993f49dc6078db2de52e8cea303407234 not found: ID does not exist"
Sep 07 11:53:32.346667 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.346622    2580 scope.go:117] "RemoveContainer" containerID="183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec"
Sep 07 11:53:32.346832 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.346812    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec"} err="failed to get container status \"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec\": rpc error: code = NotFound desc = could not find container \"183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec\": container with ID starting with 183044dc966988a7f763dcb82d3618cc885c3211a5153c8238f902009cbf2aec not found: ID does not exist"
Sep 07 11:53:32.346884 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.346833    2580 scope.go:117] "RemoveContainer" containerID="bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac"
Sep 07 11:53:32.347031 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.347017    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac"} err="failed to get container status \"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac\": rpc error: code = NotFound desc = could not find container \"bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac\": container with ID starting with bcabbeeb73e0785066d1c197089acfcb69af8be9d81c09b9d92df05b8057aaac not found: ID does not exist"
Sep 07 11:53:32.347077 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.347031    2580 scope.go:117] "RemoveContainer" containerID="7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f"
Sep 07 11:53:32.347244 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.347229    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f"} err="failed to get container status \"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f\": rpc error: code = NotFound desc = could not find container \"7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f\": container with ID starting with 7341bf3ca4acf1ae9df18f15d005994fddd0fe00bbad50e2c5c893be3a4f032f not found: ID does not exist"
Sep 07 11:53:32.350019 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.350004    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.352661 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.352642    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 07 11:53:32.352772 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.352756    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 07 11:53:32.352836 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.352763    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 07 11:53:32.352913 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.352884    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7k7hqgdd3o3tq\""
Sep 07 11:53:32.352913 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.352894    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 07 11:53:32.353031 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.352951    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 07 11:53:32.353361 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.353346    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 07 11:53:32.353443 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.353408    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 07 11:53:32.353443 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.353422    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 07 11:53:32.353547 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.353536    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 07 11:53:32.353732 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.353714    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 07 11:53:32.353806 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.353729    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 07 11:53:32.354373 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.354355    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 07 11:53:32.354545 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.354531    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 07 11:53:32.355171 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.355154    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xdg5b\""
Sep 07 11:53:32.356113 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.356093    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 07 11:53:32.363507 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.363482    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 07 11:53:32.364128 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.364106    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 11:53:32.470613 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470582    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.470613 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470618    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.470814 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.470814 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.470814 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470730    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.470814 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.470814 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470772    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb7tl\" (UniqueName: \"kubernetes.io/projected/30122c97-d13e-40c9-8dff-d6ef59cc6edd-kube-api-access-hb7tl\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.470814 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-config\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.470984 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470835    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.470984 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470854    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.470984 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470874    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.470984 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.470984 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.470984 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470933    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/30122c97-d13e-40c9-8dff-d6ef59cc6edd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.470984 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/30122c97-d13e-40c9-8dff-d6ef59cc6edd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.470984 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470966    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.470984 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.470979    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-web-config\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.471259 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.471043    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.471259 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.471059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/30122c97-d13e-40c9-8dff-d6ef59cc6edd-config-out\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.471259 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.471074    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572044 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.571955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572044 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.571999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb7tl\" (UniqueName: \"kubernetes.io/projected/30122c97-d13e-40c9-8dff-d6ef59cc6edd-kube-api-access-hb7tl\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572044 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-config\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572044 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572075    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572109    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572136    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572156    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572187    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/30122c97-d13e-40c9-8dff-d6ef59cc6edd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572257    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/30122c97-d13e-40c9-8dff-d6ef59cc6edd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572308    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572335    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-web-config\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/30122c97-d13e-40c9-8dff-d6ef59cc6edd-config-out\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572543    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572596    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.572648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.572594    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/30122c97-d13e-40c9-8dff-d6ef59cc6edd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.573528 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.573350    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.574690 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.573948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.574690 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.574504    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.575188 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.575160    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.575298 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.575160    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.575371 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.575324    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-config\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.576548 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.575924    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/30122c97-d13e-40c9-8dff-d6ef59cc6edd-config-out\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.576548 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.576147    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.576548 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.576282    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.577320 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.576791    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.577320 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.577053    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.577320 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.577291    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.577320 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.577296    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/30122c97-d13e-40c9-8dff-d6ef59cc6edd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.577530 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.577509    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.577894 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.577868    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/30122c97-d13e-40c9-8dff-d6ef59cc6edd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.577997 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.577977    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.578037 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.577980    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.578154 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.578138    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/30122c97-d13e-40c9-8dff-d6ef59cc6edd-web-config\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.580853 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.580831    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb7tl\" (UniqueName: \"kubernetes.io/projected/30122c97-d13e-40c9-8dff-d6ef59cc6edd-kube-api-access-hb7tl\") pod \"prometheus-k8s-0\" (UID: \"30122c97-d13e-40c9-8dff-d6ef59cc6edd\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.660068 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.660036    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:32.940236 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:32.940185    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 11:53:32.943457 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:53:32.943431    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30122c97_d13e_40c9_8dff_d6ef59cc6edd.slice/crio-4aa0a64467670be80c75dd5865fe21a562adc14504b1676a35f80b1293f4b1f4 WatchSource:0}: Error finding container 4aa0a64467670be80c75dd5865fe21a562adc14504b1676a35f80b1293f4b1f4: Status 404 returned error can't find the container with id 4aa0a64467670be80c75dd5865fe21a562adc14504b1676a35f80b1293f4b1f4
Sep 07 11:53:33.295959 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:33.295875    2580 generic.go:356] "Generic (PLEG): container finished" podID="30122c97-d13e-40c9-8dff-d6ef59cc6edd" containerID="5c6cd7ed1e577bcd6d563d1e3d5472b21083216eb21d2272d4fc44b13fe38cba" exitCode=0
Sep 07 11:53:33.295959 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:33.295913    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"30122c97-d13e-40c9-8dff-d6ef59cc6edd","Type":"ContainerDied","Data":"5c6cd7ed1e577bcd6d563d1e3d5472b21083216eb21d2272d4fc44b13fe38cba"}
Sep 07 11:53:33.296172 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:33.295964    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"30122c97-d13e-40c9-8dff-d6ef59cc6edd","Type":"ContainerStarted","Data":"4aa0a64467670be80c75dd5865fe21a562adc14504b1676a35f80b1293f4b1f4"}
Sep 07 11:53:34.009316 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:34.009278    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3612b76-503b-42e9-895d-87969c738012" path="/var/lib/kubelet/pods/e3612b76-503b-42e9-895d-87969c738012/volumes"
Sep 07 11:53:34.303062 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:34.302976    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"30122c97-d13e-40c9-8dff-d6ef59cc6edd","Type":"ContainerStarted","Data":"7ec0f036f143b7a9c6e38d3cfb617f8a40aabc69d9821922265bbbd2fd820b83"}
Sep 07 11:53:34.303062 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:34.303048    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"30122c97-d13e-40c9-8dff-d6ef59cc6edd","Type":"ContainerStarted","Data":"4b82aa61b08ea7d50c8bdc010cf5efb3a5386b34c1cb24ac86f8e52854d22ce8"}
Sep 07 11:53:34.303300 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:34.303095    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"30122c97-d13e-40c9-8dff-d6ef59cc6edd","Type":"ContainerStarted","Data":"fa1c673466af6948e96f63aa8301035c09578d557708b1123896983d20961dc6"}
Sep 07 11:53:34.303300 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:34.303130    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"30122c97-d13e-40c9-8dff-d6ef59cc6edd","Type":"ContainerStarted","Data":"94fa71f1df555681fc92330e1af58b4cd5f0f669b28075428c5a72e4bdafc533"}
Sep 07 11:53:34.303300 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:34.303163    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"30122c97-d13e-40c9-8dff-d6ef59cc6edd","Type":"ContainerStarted","Data":"ba733babcdb9b09df9ce457cd4e97560187c46dd2bd50dfe6b748ccc87af2060"}
Sep 07 11:53:34.303300 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:34.303224    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"30122c97-d13e-40c9-8dff-d6ef59cc6edd","Type":"ContainerStarted","Data":"70b181246b2995c9a06c33f668c0cbf0759902e0d0a30cf310f95368dc445bea"}
Sep 07 11:53:34.330837 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:34.330782    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.313182634 podStartE2EDuration="2.330760594s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.57796ms" isStatefulPod=false podCreationTimestamp="2026-09-07 11:53:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:53:34.328499008 +0000 UTC m=+368.824388436" watchObservedRunningTime="2026-09-07 11:53:34.330760594 +0000 UTC m=+368.826650017"
Sep 07 11:53:37.660440 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:37.660393    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:53:39.934639 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:39.934606    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs\") pod \"network-metrics-daemon-4dgj8\" (UID: \"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f\") " pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:53:39.936850 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:39.936831    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e4e91b4-23e4-4903-8e2d-cf1d2145b51f-metrics-certs\") pod \"network-metrics-daemon-4dgj8\" (UID: \"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f\") " pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:53:40.107592 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:40.107558    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gtblx\""
Sep 07 11:53:40.115557 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:40.115537    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dgj8"
Sep 07 11:53:40.386457 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:40.386424    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4dgj8"]
Sep 07 11:53:40.390708 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:53:40.390678    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e4e91b4_23e4_4903_8e2d_cf1d2145b51f.slice/crio-ae7da16da660817863771795243a5e5c34be98be2759f177fd015ca80e74ee79 WatchSource:0}: Error finding container ae7da16da660817863771795243a5e5c34be98be2759f177fd015ca80e74ee79: Status 404 returned error can't find the container with id ae7da16da660817863771795243a5e5c34be98be2759f177fd015ca80e74ee79
Sep 07 11:53:40.392488 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:40.392473    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 11:53:41.329648 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:41.329618    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4dgj8" event={"ID":"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f","Type":"ContainerStarted","Data":"ae7da16da660817863771795243a5e5c34be98be2759f177fd015ca80e74ee79"}
Sep 07 11:53:42.334806 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:42.334766    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4dgj8" event={"ID":"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f","Type":"ContainerStarted","Data":"cb26fb0850766027d014d762f17f1ffceedf2567c33faa40281c7414df09c385"}
Sep 07 11:53:42.335271 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:42.334850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4dgj8" event={"ID":"7e4e91b4-23e4-4903-8e2d-cf1d2145b51f","Type":"ContainerStarted","Data":"c6bc0ddd7eaab51cd8c46d8be9eff3fcf877b615830c51b33b633686da9d0325"}
Sep 07 11:53:42.351880 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:53:42.351825    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4dgj8" podStartSLOduration=375.382502638 podStartE2EDuration="6m16.351806528s" totalImagesPullingTime="969.30389ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:53:42.350789108 +0000 UTC m=+376.846678531" watchObservedRunningTime="2026-09-07 11:53:42.351806528 +0000 UTC m=+376.847695952"
Sep 07 11:54:32.660550 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:54:32.660509    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:54:32.675980 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:54:32.675951    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:54:33.528855 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:54:33.528828    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 11:57:01.114714 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.114679    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"]
Sep 07 11:57:01.117754 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.117731    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.120662 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.120639    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 07 11:57:01.120874 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.120859    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 07 11:57:01.121216 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.121186    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 07 11:57:01.121271 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.121217    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 07 11:57:01.121439 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.121427    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qnxxt\""
Sep 07 11:57:01.138842 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.138797    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"]
Sep 07 11:57:01.150150 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.150116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd7w8\" (UniqueName: \"kubernetes.io/projected/77d29c3a-d28b-4d5b-8aec-29d25c6750af-kube-api-access-xd7w8\") pod \"opendatahub-operator-controller-manager-648db6df7b-2r8zx\" (UID: \"77d29c3a-d28b-4d5b-8aec-29d25c6750af\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.150305 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.150156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77d29c3a-d28b-4d5b-8aec-29d25c6750af-tmp\") pod \"opendatahub-operator-controller-manager-648db6df7b-2r8zx\" (UID: \"77d29c3a-d28b-4d5b-8aec-29d25c6750af\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.150305 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.150244    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/77d29c3a-d28b-4d5b-8aec-29d25c6750af-apiservice-cert\") pod \"opendatahub-operator-controller-manager-648db6df7b-2r8zx\" (UID: \"77d29c3a-d28b-4d5b-8aec-29d25c6750af\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.150305 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.150281    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/77d29c3a-d28b-4d5b-8aec-29d25c6750af-manifests\") pod \"opendatahub-operator-controller-manager-648db6df7b-2r8zx\" (UID: \"77d29c3a-d28b-4d5b-8aec-29d25c6750af\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.150305 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.150304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77d29c3a-d28b-4d5b-8aec-29d25c6750af-webhook-cert\") pod \"opendatahub-operator-controller-manager-648db6df7b-2r8zx\" (UID: \"77d29c3a-d28b-4d5b-8aec-29d25c6750af\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.251159 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.251116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd7w8\" (UniqueName: \"kubernetes.io/projected/77d29c3a-d28b-4d5b-8aec-29d25c6750af-kube-api-access-xd7w8\") pod \"opendatahub-operator-controller-manager-648db6df7b-2r8zx\" (UID: \"77d29c3a-d28b-4d5b-8aec-29d25c6750af\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.251159 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.251164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77d29c3a-d28b-4d5b-8aec-29d25c6750af-tmp\") pod \"opendatahub-operator-controller-manager-648db6df7b-2r8zx\" (UID: \"77d29c3a-d28b-4d5b-8aec-29d25c6750af\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.251441 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.251230    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/77d29c3a-d28b-4d5b-8aec-29d25c6750af-apiservice-cert\") pod \"opendatahub-operator-controller-manager-648db6df7b-2r8zx\" (UID: \"77d29c3a-d28b-4d5b-8aec-29d25c6750af\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.251441 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.251268    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/77d29c3a-d28b-4d5b-8aec-29d25c6750af-manifests\") pod \"opendatahub-operator-controller-manager-648db6df7b-2r8zx\" (UID: \"77d29c3a-d28b-4d5b-8aec-29d25c6750af\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.251441 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.251299    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77d29c3a-d28b-4d5b-8aec-29d25c6750af-webhook-cert\") pod \"opendatahub-operator-controller-manager-648db6df7b-2r8zx\" (UID: \"77d29c3a-d28b-4d5b-8aec-29d25c6750af\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.251710 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.251681    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/77d29c3a-d28b-4d5b-8aec-29d25c6750af-manifests\") pod \"opendatahub-operator-controller-manager-648db6df7b-2r8zx\" (UID: \"77d29c3a-d28b-4d5b-8aec-29d25c6750af\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.253520 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.253498    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77d29c3a-d28b-4d5b-8aec-29d25c6750af-tmp\") pod \"opendatahub-operator-controller-manager-648db6df7b-2r8zx\" (UID: \"77d29c3a-d28b-4d5b-8aec-29d25c6750af\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.253775 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.253756    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/77d29c3a-d28b-4d5b-8aec-29d25c6750af-apiservice-cert\") pod \"opendatahub-operator-controller-manager-648db6df7b-2r8zx\" (UID: \"77d29c3a-d28b-4d5b-8aec-29d25c6750af\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.253836 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.253756    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77d29c3a-d28b-4d5b-8aec-29d25c6750af-webhook-cert\") pod \"opendatahub-operator-controller-manager-648db6df7b-2r8zx\" (UID: \"77d29c3a-d28b-4d5b-8aec-29d25c6750af\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.262042 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.262019    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd7w8\" (UniqueName: \"kubernetes.io/projected/77d29c3a-d28b-4d5b-8aec-29d25c6750af-kube-api-access-xd7w8\") pod \"opendatahub-operator-controller-manager-648db6df7b-2r8zx\" (UID: \"77d29c3a-d28b-4d5b-8aec-29d25c6750af\") " pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.428213 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.428163    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:01.704712 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:01.704631    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"]
Sep 07 11:57:01.707753 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:57:01.707728    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d29c3a_d28b_4d5b_8aec_29d25c6750af.slice/crio-ce78bdc249f83960ac3bf56c7b5180b2d6155670e76afc1134d0b80c3157e766 WatchSource:0}: Error finding container ce78bdc249f83960ac3bf56c7b5180b2d6155670e76afc1134d0b80c3157e766: Status 404 returned error can't find the container with id ce78bdc249f83960ac3bf56c7b5180b2d6155670e76afc1134d0b80c3157e766
Sep 07 11:57:02.042337 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:02.042249    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx" event={"ID":"77d29c3a-d28b-4d5b-8aec-29d25c6750af","Type":"ContainerStarted","Data":"ce78bdc249f83960ac3bf56c7b5180b2d6155670e76afc1134d0b80c3157e766"}
Sep 07 11:57:05.056361 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:05.056322    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx" event={"ID":"77d29c3a-d28b-4d5b-8aec-29d25c6750af","Type":"ContainerStarted","Data":"ca52b926e4e133e9a94751724649a952bf92c76fb13a574fbf97294f982e825f"}
Sep 07 11:57:06.060840 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:06.060749    2580 generic.go:356] "Generic (PLEG): container finished" podID="77d29c3a-d28b-4d5b-8aec-29d25c6750af" containerID="ca52b926e4e133e9a94751724649a952bf92c76fb13a574fbf97294f982e825f" exitCode=0
Sep 07 11:57:06.061345 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:06.060791    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx" event={"ID":"77d29c3a-d28b-4d5b-8aec-29d25c6750af","Type":"ContainerDied","Data":"ca52b926e4e133e9a94751724649a952bf92c76fb13a574fbf97294f982e825f"}
Sep 07 11:57:07.068371 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:07.068334    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx" event={"ID":"77d29c3a-d28b-4d5b-8aec-29d25c6750af","Type":"ContainerStarted","Data":"bb0b92b9a1128874b94a164ba654ee1f291fa174972dc31736c2db19b6694768"}
Sep 07 11:57:07.068781 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:07.068481    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:07.096631 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:07.096577    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx" podStartSLOduration=1.7859064569999998 podStartE2EDuration="6.096560432s" totalImagesPullingTime="3.2286772s" totalInitContainerRuntime="1.081976775s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:57:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:57:07.095436954 +0000 UTC m=+581.591326378" watchObservedRunningTime="2026-09-07 11:57:07.096560432 +0000 UTC m=+581.592449856"
Sep 07 11:57:18.074890 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:18.074858    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-648db6df7b-2r8zx"
Sep 07 11:57:20.789859 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:20.789817    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"]
Sep 07 11:57:20.792983 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:20.792961    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"
Sep 07 11:57:20.795426 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:20.795403    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 07 11:57:20.796537 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:20.796513    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 07 11:57:20.797102 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:20.797082    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 07 11:57:20.797256 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:20.797086    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 07 11:57:20.797369 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:20.797086    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tslk4\""
Sep 07 11:57:20.797369 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:20.797091    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 07 11:57:20.803809 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:20.803786    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"]
Sep 07 11:57:20.903486 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:20.903454    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d28abc0d-55e9-49b2-8396-059e2a514373-cert\") pod \"lws-controller-manager-59499dc5ff-vc8vh\" (UID: \"d28abc0d-55e9-49b2-8396-059e2a514373\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"
Sep 07 11:57:20.903486 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:20.903496    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdkpp\" (UniqueName: \"kubernetes.io/projected/d28abc0d-55e9-49b2-8396-059e2a514373-kube-api-access-cdkpp\") pod \"lws-controller-manager-59499dc5ff-vc8vh\" (UID: \"d28abc0d-55e9-49b2-8396-059e2a514373\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"
Sep 07 11:57:20.903703 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:20.903528    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d28abc0d-55e9-49b2-8396-059e2a514373-manager-config\") pod \"lws-controller-manager-59499dc5ff-vc8vh\" (UID: \"d28abc0d-55e9-49b2-8396-059e2a514373\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"
Sep 07 11:57:20.903703 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:20.903561    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d28abc0d-55e9-49b2-8396-059e2a514373-metrics-cert\") pod \"lws-controller-manager-59499dc5ff-vc8vh\" (UID: \"d28abc0d-55e9-49b2-8396-059e2a514373\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"
Sep 07 11:57:21.004817 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:21.004776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d28abc0d-55e9-49b2-8396-059e2a514373-manager-config\") pod \"lws-controller-manager-59499dc5ff-vc8vh\" (UID: \"d28abc0d-55e9-49b2-8396-059e2a514373\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"
Sep 07 11:57:21.004817 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:21.004815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d28abc0d-55e9-49b2-8396-059e2a514373-metrics-cert\") pod \"lws-controller-manager-59499dc5ff-vc8vh\" (UID: \"d28abc0d-55e9-49b2-8396-059e2a514373\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"
Sep 07 11:57:21.005019 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:21.004867    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d28abc0d-55e9-49b2-8396-059e2a514373-cert\") pod \"lws-controller-manager-59499dc5ff-vc8vh\" (UID: \"d28abc0d-55e9-49b2-8396-059e2a514373\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"
Sep 07 11:57:21.005019 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:21.004891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdkpp\" (UniqueName: \"kubernetes.io/projected/d28abc0d-55e9-49b2-8396-059e2a514373-kube-api-access-cdkpp\") pod \"lws-controller-manager-59499dc5ff-vc8vh\" (UID: \"d28abc0d-55e9-49b2-8396-059e2a514373\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"
Sep 07 11:57:21.005430 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:21.005406    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d28abc0d-55e9-49b2-8396-059e2a514373-manager-config\") pod \"lws-controller-manager-59499dc5ff-vc8vh\" (UID: \"d28abc0d-55e9-49b2-8396-059e2a514373\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"
Sep 07 11:57:21.007335 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:21.007306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d28abc0d-55e9-49b2-8396-059e2a514373-metrics-cert\") pod \"lws-controller-manager-59499dc5ff-vc8vh\" (UID: \"d28abc0d-55e9-49b2-8396-059e2a514373\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"
Sep 07 11:57:21.007461 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:21.007436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d28abc0d-55e9-49b2-8396-059e2a514373-cert\") pod \"lws-controller-manager-59499dc5ff-vc8vh\" (UID: \"d28abc0d-55e9-49b2-8396-059e2a514373\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"
Sep 07 11:57:21.012498 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:21.012469    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdkpp\" (UniqueName: \"kubernetes.io/projected/d28abc0d-55e9-49b2-8396-059e2a514373-kube-api-access-cdkpp\") pod \"lws-controller-manager-59499dc5ff-vc8vh\" (UID: \"d28abc0d-55e9-49b2-8396-059e2a514373\") " pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"
Sep 07 11:57:21.102097 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:21.102014    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"
Sep 07 11:57:21.374673 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:21.374649    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"]
Sep 07 11:57:21.377172 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:57:21.377141    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd28abc0d_55e9_49b2_8396_059e2a514373.slice/crio-a06ebd709391a785848cd19fc3bcaf0222a5a9ed58e9c6373cc8060c35759bb3 WatchSource:0}: Error finding container a06ebd709391a785848cd19fc3bcaf0222a5a9ed58e9c6373cc8060c35759bb3: Status 404 returned error can't find the container with id a06ebd709391a785848cd19fc3bcaf0222a5a9ed58e9c6373cc8060c35759bb3
Sep 07 11:57:22.126894 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:22.126853    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh" event={"ID":"d28abc0d-55e9-49b2-8396-059e2a514373","Type":"ContainerStarted","Data":"a06ebd709391a785848cd19fc3bcaf0222a5a9ed58e9c6373cc8060c35759bb3"}
Sep 07 11:57:24.135100 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:24.135069    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh" event={"ID":"d28abc0d-55e9-49b2-8396-059e2a514373","Type":"ContainerStarted","Data":"f7216fd37a003e95ee01ee4df8f11ba8aed79af9dbe51083ff6c6b5f125e9b14"}
Sep 07 11:57:24.135492 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:24.135181    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"
Sep 07 11:57:24.154787 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:24.154738    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh" podStartSLOduration=1.688262511 podStartE2EDuration="4.154721473s" totalImagesPullingTime="2.466458962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:57:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:57:24.152728277 +0000 UTC m=+598.648617704" watchObservedRunningTime="2026-09-07 11:57:24.154721473 +0000 UTC m=+598.650610893"
Sep 07 11:57:26.590499 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.590464    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"]
Sep 07 11:57:26.595180 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.595153    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.597813 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.597790    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 07 11:57:26.597966 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.597839    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 07 11:57:26.597966 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.597792    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 07 11:57:26.597966 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.597913    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7qm6f\""
Sep 07 11:57:26.605952 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.605929    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"]
Sep 07 11:57:26.644750 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.644715    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"]
Sep 07 11:57:26.647743 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.647726    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.651522 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.651494    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.651670 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.651530    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.651670 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.651604    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.651670 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.651638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b71d3665-3a1d-4347-b510-f0d8b930e243-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.651833 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.651683    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.651833 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.651723    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.651833 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.651775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wfcv\" (UniqueName: \"kubernetes.io/projected/b71d3665-3a1d-4347-b510-f0d8b930e243-kube-api-access-7wfcv\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.651833 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.651820    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.652021 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.651881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.659713 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.659689    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"]
Sep 07 11:57:26.752535 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752424    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a3f497d-1b05-434f-a4b4-69dc09a87643-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.752535 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752476    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a3f497d-1b05-434f-a4b4-69dc09a87643-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.752535 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.752535 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752530    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a3f497d-1b05-434f-a4b4-69dc09a87643-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.765459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752547    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz4j2\" (UniqueName: \"kubernetes.io/projected/7a3f497d-1b05-434f-a4b4-69dc09a87643-kube-api-access-tz4j2\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.765459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752566    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.765459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wfcv\" (UniqueName: \"kubernetes.io/projected/b71d3665-3a1d-4347-b510-f0d8b930e243-kube-api-access-7wfcv\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.765459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.765459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752651    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a3f497d-1b05-434f-a4b4-69dc09a87643-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.765459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.765459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752709    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a3f497d-1b05-434f-a4b4-69dc09a87643-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.765459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752740    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.765459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752769    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.765459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752792    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a3f497d-1b05-434f-a4b4-69dc09a87643-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.765459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752872    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.765459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752913    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b71d3665-3a1d-4347-b510-f0d8b930e243-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.765459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a3f497d-1b05-434f-a4b4-69dc09a87643-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.765459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.752967    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a3f497d-1b05-434f-a4b4-69dc09a87643-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.766186 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.753000    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.766186 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.753078    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.766186 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.753156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.766186 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.753180    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.766186 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.753537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b71d3665-3a1d-4347-b510-f0d8b930e243-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.766186 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.755058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.766186 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.755324    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.766186 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.760700    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.766186 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.760855    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wfcv\" (UniqueName: \"kubernetes.io/projected/b71d3665-3a1d-4347-b510-f0d8b930e243-kube-api-access-7wfcv\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.854238 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.854135    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a3f497d-1b05-434f-a4b4-69dc09a87643-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.854238 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.854187    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a3f497d-1b05-434f-a4b4-69dc09a87643-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.854459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.854253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a3f497d-1b05-434f-a4b4-69dc09a87643-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.854459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.854289    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a3f497d-1b05-434f-a4b4-69dc09a87643-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.854459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.854315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a3f497d-1b05-434f-a4b4-69dc09a87643-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.854459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.854347    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a3f497d-1b05-434f-a4b4-69dc09a87643-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.854459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.854369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a3f497d-1b05-434f-a4b4-69dc09a87643-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.854459 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.854400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a3f497d-1b05-434f-a4b4-69dc09a87643-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.854753 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.854640    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a3f497d-1b05-434f-a4b4-69dc09a87643-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.854753 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.854655    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a3f497d-1b05-434f-a4b4-69dc09a87643-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.854753 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.854711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz4j2\" (UniqueName: \"kubernetes.io/projected/7a3f497d-1b05-434f-a4b4-69dc09a87643-kube-api-access-tz4j2\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.854753 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.854742    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a3f497d-1b05-434f-a4b4-69dc09a87643-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.854949 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.854907    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a3f497d-1b05-434f-a4b4-69dc09a87643-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.855360 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.855333    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a3f497d-1b05-434f-a4b4-69dc09a87643-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.856770 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.856746    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a3f497d-1b05-434f-a4b4-69dc09a87643-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.856941 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.856924    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a3f497d-1b05-434f-a4b4-69dc09a87643-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.862899 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.862876    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a3f497d-1b05-434f-a4b4-69dc09a87643-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.863278 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.863261    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz4j2\" (UniqueName: \"kubernetes.io/projected/7a3f497d-1b05-434f-a4b4-69dc09a87643-kube-api-access-tz4j2\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff\" (UID: \"7a3f497d-1b05-434f-a4b4-69dc09a87643\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:26.906252 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.906215    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:26.958490 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:26.958461    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:27.179403 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:27.179314    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"]
Sep 07 11:57:27.183301 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:57:27.183275    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb71d3665_3a1d_4347_b510_f0d8b930e243.slice/crio-4cb8d3d7335464c85d1f6e76a2cbfbe132d71da25f07b70dbd46ead36a8b2aef WatchSource:0}: Error finding container 4cb8d3d7335464c85d1f6e76a2cbfbe132d71da25f07b70dbd46ead36a8b2aef: Status 404 returned error can't find the container with id 4cb8d3d7335464c85d1f6e76a2cbfbe132d71da25f07b70dbd46ead36a8b2aef
Sep 07 11:57:27.235042 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:27.235015    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"]
Sep 07 11:57:27.237459 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:57:27.237428    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a3f497d_1b05_434f_a4b4_69dc09a87643.slice/crio-ef611dd1625c68ae313bb751fe472cb62e7cbb3b28fc1f9e1e2b5f5888872ecc WatchSource:0}: Error finding container ef611dd1625c68ae313bb751fe472cb62e7cbb3b28fc1f9e1e2b5f5888872ecc: Status 404 returned error can't find the container with id ef611dd1625c68ae313bb751fe472cb62e7cbb3b28fc1f9e1e2b5f5888872ecc
Sep 07 11:57:28.153499 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:28.153455    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff" event={"ID":"7a3f497d-1b05-434f-a4b4-69dc09a87643","Type":"ContainerStarted","Data":"ef611dd1625c68ae313bb751fe472cb62e7cbb3b28fc1f9e1e2b5f5888872ecc"}
Sep 07 11:57:28.155672 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:28.155644    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q" event={"ID":"b71d3665-3a1d-4347-b510-f0d8b930e243","Type":"ContainerStarted","Data":"4cb8d3d7335464c85d1f6e76a2cbfbe132d71da25f07b70dbd46ead36a8b2aef"}
Sep 07 11:57:30.109696 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:30.109651    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 11:57:30.109995 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:30.109736    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 11:57:30.109995 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:30.109771    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 11:57:30.117384 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:30.117351    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 11:57:30.117483 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:30.117418    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 11:57:30.117483 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:30.117451    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 11:57:31.171217 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:31.171171    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q" event={"ID":"b71d3665-3a1d-4347-b510-f0d8b930e243","Type":"ContainerStarted","Data":"ec1f15487b34e4896d3eb8dc7d40cea1ba83d88eb57e2e6c7e167565336ec536"}
Sep 07 11:57:31.172897 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:31.172862    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff" event={"ID":"7a3f497d-1b05-434f-a4b4-69dc09a87643","Type":"ContainerStarted","Data":"d7912abfb661ae3be46d85c62e6c76b4012643f0f523423cb1a28095a126a5ce"}
Sep 07 11:57:31.190031 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:31.189969    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q" podStartSLOduration=2.265850842 podStartE2EDuration="5.18994542s" totalImagesPullingTime="2.924094578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:57:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:57:31.188765547 +0000 UTC m=+605.684654972" watchObservedRunningTime="2026-09-07 11:57:31.18994542 +0000 UTC m=+605.685834844"
Sep 07 11:57:31.210372 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:31.210310    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff" podStartSLOduration=2.332640222 podStartE2EDuration="5.210291495s" totalImagesPullingTime="2.877651273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:57:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:57:31.208771667 +0000 UTC m=+605.704661100" watchObservedRunningTime="2026-09-07 11:57:31.210291495 +0000 UTC m=+605.706180918"
Sep 07 11:57:31.907351 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:31.907314    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:31.908639 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:31.908611    2580 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.16:15021/healthz/ready\": dial tcp 10.134.0.16:15021: connect: connection refused" start-of-body=
Sep 07 11:57:31.908770 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:31.908669    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q" podUID="b71d3665-3a1d-4347-b510-f0d8b930e243" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.16:15021/healthz/ready\": dial tcp 10.134.0.16:15021: connect: connection refused"
Sep 07 11:57:31.959436 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:31.959397    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:31.964144 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:31.964120    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:32.182336 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:32.182245    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:32.183652 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:32.183624    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff"
Sep 07 11:57:32.230886 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:32.230850    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"]
Sep 07 11:57:32.907635 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:32.907594    2580 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.16:15021/healthz/ready\": dial tcp 10.134.0.16:15021: connect: connection refused" start-of-body=
Sep 07 11:57:32.907784 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:32.907685    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q" podUID="b71d3665-3a1d-4347-b510-f0d8b930e243" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.16:15021/healthz/ready\": dial tcp 10.134.0.16:15021: connect: connection refused"
Sep 07 11:57:33.907162 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:33.907129    2580 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.16:15021/healthz/ready\": dial tcp 10.134.0.16:15021: connect: connection refused" start-of-body=
Sep 07 11:57:33.907579 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:33.907224    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q" podUID="b71d3665-3a1d-4347-b510-f0d8b930e243" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.16:15021/healthz/ready\": dial tcp 10.134.0.16:15021: connect: connection refused"
Sep 07 11:57:34.188554 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:34.188449    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q" podUID="b71d3665-3a1d-4347-b510-f0d8b930e243" containerName="istio-proxy" containerID="cri-o://ec1f15487b34e4896d3eb8dc7d40cea1ba83d88eb57e2e6c7e167565336ec536" gracePeriod=30
Sep 07 11:57:35.140758 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:35.140729    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59499dc5ff-vc8vh"
Sep 07 11:57:37.740989 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.740953    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"]
Sep 07 11:57:37.745534 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.745517    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.748387 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.748350    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 07 11:57:37.748387 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.748373    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 07 11:57:37.748571 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.748365    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 07 11:57:37.748571 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.748495    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vmvtw\""
Sep 07 11:57:37.757893 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.757868    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"]
Sep 07 11:57:37.848801 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.848760    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.848801 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.848800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.849018 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.848830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7c78\" (UniqueName: \"kubernetes.io/projected/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-kube-api-access-w7c78\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.849018 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.848872    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.849018 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.848891    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.849018 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.848920    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.849018 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.848955    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.950356 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.950323    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.950433 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.950360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.950433 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.950386    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.950433 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.950414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.950565 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.950456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.950565 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.950471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.950653 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.950629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7c78\" (UniqueName: \"kubernetes.io/projected/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-kube-api-access-w7c78\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.951266 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.951240    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.952704 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.952681    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.952885 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.952859    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.953159 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.953140    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.953218 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.953153    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.958121 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.958097    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:37.958562 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:37.958540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7c78\" (UniqueName: \"kubernetes.io/projected/83e0ccb3-22c8-478a-9635-1928e6bcc2c7-kube-api-access-w7c78\") pod \"istiod-openshift-gateway-5677ccb49b-x5jvb\" (UID: \"83e0ccb3-22c8-478a-9635-1928e6bcc2c7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:38.055319 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:38.055222    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:38.331272 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:38.331248    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"]
Sep 07 11:57:38.333984 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:57:38.333956    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83e0ccb3_22c8_478a_9635_1928e6bcc2c7.slice/crio-2fb754a3b16eea14c259dd36836ec9a60830ac645878c99b2f2efadf062be0ec WatchSource:0}: Error finding container 2fb754a3b16eea14c259dd36836ec9a60830ac645878c99b2f2efadf062be0ec: Status 404 returned error can't find the container with id 2fb754a3b16eea14c259dd36836ec9a60830ac645878c99b2f2efadf062be0ec
Sep 07 11:57:39.210144 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.210109    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb" event={"ID":"83e0ccb3-22c8-478a-9635-1928e6bcc2c7","Type":"ContainerStarted","Data":"2fb754a3b16eea14c259dd36836ec9a60830ac645878c99b2f2efadf062be0ec"}
Sep 07 11:57:39.456791 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.456729    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:39.563809 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.563770    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-data\") pod \"b71d3665-3a1d-4347-b510-f0d8b930e243\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") "
Sep 07 11:57:39.564001 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.563837    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-podinfo\") pod \"b71d3665-3a1d-4347-b510-f0d8b930e243\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") "
Sep 07 11:57:39.564001 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.563860    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wfcv\" (UniqueName: \"kubernetes.io/projected/b71d3665-3a1d-4347-b510-f0d8b930e243-kube-api-access-7wfcv\") pod \"b71d3665-3a1d-4347-b510-f0d8b930e243\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") "
Sep 07 11:57:39.564001 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.563892    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-credential-socket\") pod \"b71d3665-3a1d-4347-b510-f0d8b930e243\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") "
Sep 07 11:57:39.564001 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.563923    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-envoy\") pod \"b71d3665-3a1d-4347-b510-f0d8b930e243\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") "
Sep 07 11:57:39.564001 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.563952    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b71d3665-3a1d-4347-b510-f0d8b930e243-istiod-ca-cert\") pod \"b71d3665-3a1d-4347-b510-f0d8b930e243\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") "
Sep 07 11:57:39.564001 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.563974    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-workload-socket\") pod \"b71d3665-3a1d-4347-b510-f0d8b930e243\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") "
Sep 07 11:57:39.564001 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.564002    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-workload-certs\") pod \"b71d3665-3a1d-4347-b510-f0d8b930e243\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") "
Sep 07 11:57:39.565037 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.564029    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-token\") pod \"b71d3665-3a1d-4347-b510-f0d8b930e243\" (UID: \"b71d3665-3a1d-4347-b510-f0d8b930e243\") "
Sep 07 11:57:39.565037 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.564696    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b71d3665-3a1d-4347-b510-f0d8b930e243" (UID: "b71d3665-3a1d-4347-b510-f0d8b930e243"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:57:39.565037 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.564747    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-data" (OuterVolumeSpecName: "istio-data") pod "b71d3665-3a1d-4347-b510-f0d8b930e243" (UID: "b71d3665-3a1d-4347-b510-f0d8b930e243"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:57:39.565037 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.564821    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b71d3665-3a1d-4347-b510-f0d8b930e243" (UID: "b71d3665-3a1d-4347-b510-f0d8b930e243"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:57:39.565037 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.564866    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b71d3665-3a1d-4347-b510-f0d8b930e243" (UID: "b71d3665-3a1d-4347-b510-f0d8b930e243"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:57:39.565037 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.565015    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b71d3665-3a1d-4347-b510-f0d8b930e243-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b71d3665-3a1d-4347-b510-f0d8b930e243" (UID: "b71d3665-3a1d-4347-b510-f0d8b930e243"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:57:39.567922 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.567894    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-token" (OuterVolumeSpecName: "istio-token") pod "b71d3665-3a1d-4347-b510-f0d8b930e243" (UID: "b71d3665-3a1d-4347-b510-f0d8b930e243"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:57:39.568039 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.567920    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b71d3665-3a1d-4347-b510-f0d8b930e243-kube-api-access-7wfcv" (OuterVolumeSpecName: "kube-api-access-7wfcv") pod "b71d3665-3a1d-4347-b510-f0d8b930e243" (UID: "b71d3665-3a1d-4347-b510-f0d8b930e243"). InnerVolumeSpecName "kube-api-access-7wfcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:57:39.568177 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.568154    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b71d3665-3a1d-4347-b510-f0d8b930e243" (UID: "b71d3665-3a1d-4347-b510-f0d8b930e243"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 11:57:39.568875 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.568852    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b71d3665-3a1d-4347-b510-f0d8b930e243" (UID: "b71d3665-3a1d-4347-b510-f0d8b930e243"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 11:57:39.665147 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.665109    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-workload-certs\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:57:39.665147 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.665142    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-token\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:57:39.665147 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.665155    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-data\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:57:39.665420 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.665165    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-podinfo\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:57:39.665420 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.665176    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wfcv\" (UniqueName: \"kubernetes.io/projected/b71d3665-3a1d-4347-b510-f0d8b930e243-kube-api-access-7wfcv\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:57:39.665420 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.665188    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-credential-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:57:39.665420 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.665212    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-istio-envoy\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:57:39.665420 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.665223    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b71d3665-3a1d-4347-b510-f0d8b930e243-istiod-ca-cert\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:57:39.665420 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:39.665233    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71d3665-3a1d-4347-b510-f0d8b930e243-workload-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 11:57:40.215806 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:40.215769    2580 generic.go:356] "Generic (PLEG): container finished" podID="b71d3665-3a1d-4347-b510-f0d8b930e243" containerID="ec1f15487b34e4896d3eb8dc7d40cea1ba83d88eb57e2e6c7e167565336ec536" exitCode=0
Sep 07 11:57:40.215806 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:40.215819    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"
Sep 07 11:57:40.216347 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:40.215858    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q" event={"ID":"b71d3665-3a1d-4347-b510-f0d8b930e243","Type":"ContainerDied","Data":"ec1f15487b34e4896d3eb8dc7d40cea1ba83d88eb57e2e6c7e167565336ec536"}
Sep 07 11:57:40.216347 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:40.215940    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q" event={"ID":"b71d3665-3a1d-4347-b510-f0d8b930e243","Type":"ContainerDied","Data":"4cb8d3d7335464c85d1f6e76a2cbfbe132d71da25f07b70dbd46ead36a8b2aef"}
Sep 07 11:57:40.216347 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:40.215970    2580 scope.go:117] "RemoveContainer" containerID="ec1f15487b34e4896d3eb8dc7d40cea1ba83d88eb57e2e6c7e167565336ec536"
Sep 07 11:57:40.235069 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:40.235034    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"]
Sep 07 11:57:40.240701 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:40.240676    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcdbl5q"]
Sep 07 11:57:40.816435 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:40.816181    2580 scope.go:117] "RemoveContainer" containerID="ec1f15487b34e4896d3eb8dc7d40cea1ba83d88eb57e2e6c7e167565336ec536"
Sep 07 11:57:40.816582 ip-10-0-128-68 kubenswrapper[2580]: E0907 11:57:40.816558    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec1f15487b34e4896d3eb8dc7d40cea1ba83d88eb57e2e6c7e167565336ec536\": container with ID starting with ec1f15487b34e4896d3eb8dc7d40cea1ba83d88eb57e2e6c7e167565336ec536 not found: ID does not exist" containerID="ec1f15487b34e4896d3eb8dc7d40cea1ba83d88eb57e2e6c7e167565336ec536"
Sep 07 11:57:40.816620 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:40.816593    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec1f15487b34e4896d3eb8dc7d40cea1ba83d88eb57e2e6c7e167565336ec536"} err="failed to get container status \"ec1f15487b34e4896d3eb8dc7d40cea1ba83d88eb57e2e6c7e167565336ec536\": rpc error: code = NotFound desc = could not find container \"ec1f15487b34e4896d3eb8dc7d40cea1ba83d88eb57e2e6c7e167565336ec536\": container with ID starting with ec1f15487b34e4896d3eb8dc7d40cea1ba83d88eb57e2e6c7e167565336ec536 not found: ID does not exist"
Sep 07 11:57:40.874008 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:40.873977    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 11:57:40.874129 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:40.874051    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 11:57:41.223782 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:41.223744    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb" event={"ID":"83e0ccb3-22c8-478a-9635-1928e6bcc2c7","Type":"ContainerStarted","Data":"eb7f0ce5fe392f2587b36d5405a7d9d87ea6d80549385db67f20574c15a6c7a5"}
Sep 07 11:57:41.224277 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:41.223909    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:57:41.225634 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:41.225608    2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-x5jvb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 11:57:41.225744 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:41.225657    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb" podUID="83e0ccb3-22c8-478a-9635-1928e6bcc2c7" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 11:57:41.251109 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:41.251039    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb" podStartSLOduration=1.713830401 podStartE2EDuration="4.251016942s" totalImagesPullingTime="2.537186541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:57:41.248719428 +0000 UTC m=+615.744608863" watchObservedRunningTime="2026-09-07 11:57:41.251016942 +0000 UTC m=+615.746906366"
Sep 07 11:57:42.013924 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:42.013883    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b71d3665-3a1d-4347-b510-f0d8b930e243" path="/var/lib/kubelet/pods/b71d3665-3a1d-4347-b510-f0d8b930e243/volumes"
Sep 07 11:57:42.229133 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:57:42.229101    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-x5jvb"
Sep 07 11:59:35.669631 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.669596    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"]
Sep 07 11:59:35.670185 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.670014    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b71d3665-3a1d-4347-b510-f0d8b930e243" containerName="istio-proxy"
Sep 07 11:59:35.670185 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.670033    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b71d3665-3a1d-4347-b510-f0d8b930e243" containerName="istio-proxy"
Sep 07 11:59:35.670185 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.670109    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b71d3665-3a1d-4347-b510-f0d8b930e243" containerName="istio-proxy"
Sep 07 11:59:35.672282 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.672262    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.674867 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.674843    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-khbqn\""
Sep 07 11:59:35.684843 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.684818    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"]
Sep 07 11:59:35.778737 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.778705    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.778951 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.778744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.778951 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.778763    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.778951 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.778852    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.778951 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.778897    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.779102 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.778951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.779102 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.778997    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.779102 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.779015    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.779102 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.779049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njhkn\" (UniqueName: \"kubernetes.io/projected/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-kube-api-access-njhkn\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.880041 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.879999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njhkn\" (UniqueName: \"kubernetes.io/projected/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-kube-api-access-njhkn\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.880041 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.880043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.880322 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.880064    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.880322 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.880079    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.880322 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.880121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.880322 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.880163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.880322 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.880218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.880322 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.880254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.880322 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.880275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.880661 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.880516    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.880661 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.880601    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.880661 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.880646    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.880835 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.880811    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.880948 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.880926    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.882580 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.882553    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.882580 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.882569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.887828 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.887802    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.887917 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.887844    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njhkn\" (UniqueName: \"kubernetes.io/projected/5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952-kube-api-access-njhkn\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6pgwj\" (UID: \"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:35.982555 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:35.982473    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:36.255076 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:36.254999    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"]
Sep 07 11:59:36.258078 ip-10-0-128-68 kubenswrapper[2580]: W0907 11:59:36.258038    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f7b6385_d2f7_46a9_a1be_3f8dbf8ff952.slice/crio-22a407c7166e9e1f2ff4f57a9c439167652e34825913713015c6cef7fcb4301a WatchSource:0}: Error finding container 22a407c7166e9e1f2ff4f57a9c439167652e34825913713015c6cef7fcb4301a: Status 404 returned error can't find the container with id 22a407c7166e9e1f2ff4f57a9c439167652e34825913713015c6cef7fcb4301a
Sep 07 11:59:36.260311 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:36.260271    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 11:59:36.260398 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:36.260362    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 11:59:36.260438 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:36.260403    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 11:59:36.655824 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:36.655784    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj" event={"ID":"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952","Type":"ContainerStarted","Data":"5f16842afdb55056b6e8d779276a254c84642c963836f0ce6ee523450c55564a"}
Sep 07 11:59:36.656010 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:36.655849    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj" event={"ID":"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952","Type":"ContainerStarted","Data":"22a407c7166e9e1f2ff4f57a9c439167652e34825913713015c6cef7fcb4301a"}
Sep 07 11:59:36.677317 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:36.677267    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj" podStartSLOduration=1.677248823 podStartE2EDuration="1.677248823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:59:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:59:36.675238384 +0000 UTC m=+731.171127807" watchObservedRunningTime="2026-09-07 11:59:36.677248823 +0000 UTC m=+731.173138246"
Sep 07 11:59:36.982980 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:36.982878    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:37.988175 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:37.988144    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:38.663703 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:38.663673    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 11:59:38.664709 ip-10-0-128-68 kubenswrapper[2580]: I0907 11:59:38.664685    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 12:00:20.436915 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:20.436878    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74f97db495-g77z9"]
Sep 07 12:00:20.440094 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:20.440075    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:20.442470 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:20.442444    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 07 12:00:20.442591 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:20.442556    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8zmfq\""
Sep 07 12:00:20.442662 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:20.442588    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 07 12:00:20.442662 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:20.442649    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 07 12:00:20.447863 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:20.447838    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74f97db495-g77z9"]
Sep 07 12:00:20.555252 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:20.555192    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-webhook-cert\") pod \"maas-controller-74f97db495-g77z9\" (UID: \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\") " pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:20.555448 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:20.555269    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgdls\" (UniqueName: \"kubernetes.io/projected/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-kube-api-access-sgdls\") pod \"maas-controller-74f97db495-g77z9\" (UID: \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\") " pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:20.555448 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:20.555306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-metrics-certs\") pod \"maas-controller-74f97db495-g77z9\" (UID: \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\") " pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:20.655698 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:20.655653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-metrics-certs\") pod \"maas-controller-74f97db495-g77z9\" (UID: \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\") " pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:20.655881 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:00:20.655809    2580 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 07 12:00:20.655881 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:20.655826    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-webhook-cert\") pod \"maas-controller-74f97db495-g77z9\" (UID: \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\") " pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:20.655881 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:00:20.655874    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-metrics-certs podName:0b1521a0-83c2-41d0-a62f-e4d6b80651e6 nodeName:}" failed. No retries permitted until 2026-09-07 12:00:21.155856917 +0000 UTC m=+775.651746322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-metrics-certs") pod "maas-controller-74f97db495-g77z9" (UID: "0b1521a0-83c2-41d0-a62f-e4d6b80651e6") : secret "maas-controller-metrics-tls" not found
Sep 07 12:00:20.656047 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:20.655905    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgdls\" (UniqueName: \"kubernetes.io/projected/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-kube-api-access-sgdls\") pod \"maas-controller-74f97db495-g77z9\" (UID: \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\") " pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:20.656047 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:00:20.655923    2580 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 07 12:00:20.656047 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:00:20.655973    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-webhook-cert podName:0b1521a0-83c2-41d0-a62f-e4d6b80651e6 nodeName:}" failed. No retries permitted until 2026-09-07 12:00:21.155957567 +0000 UTC m=+775.651846971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-webhook-cert") pod "maas-controller-74f97db495-g77z9" (UID: "0b1521a0-83c2-41d0-a62f-e4d6b80651e6") : secret "maas-controller-webhook-cert" not found
Sep 07 12:00:20.663662 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:20.663635    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgdls\" (UniqueName: \"kubernetes.io/projected/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-kube-api-access-sgdls\") pod \"maas-controller-74f97db495-g77z9\" (UID: \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\") " pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:21.159675 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:21.159638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-webhook-cert\") pod \"maas-controller-74f97db495-g77z9\" (UID: \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\") " pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:21.159866 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:21.159706    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-metrics-certs\") pod \"maas-controller-74f97db495-g77z9\" (UID: \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\") " pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:21.162052 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:21.162025    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-webhook-cert\") pod \"maas-controller-74f97db495-g77z9\" (UID: \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\") " pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:21.162194 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:21.162173    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-metrics-certs\") pod \"maas-controller-74f97db495-g77z9\" (UID: \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\") " pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:21.351378 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:21.351344    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:21.618045 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:21.618020    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74f97db495-g77z9"]
Sep 07 12:00:21.620235 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:00:21.620211    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b1521a0_83c2_41d0_a62f_e4d6b80651e6.slice/crio-d5c5bc7df01fd8f52e795102f47e3da079c085c68672d9246450f3363e682bf1 WatchSource:0}: Error finding container d5c5bc7df01fd8f52e795102f47e3da079c085c68672d9246450f3363e682bf1: Status 404 returned error can't find the container with id d5c5bc7df01fd8f52e795102f47e3da079c085c68672d9246450f3363e682bf1
Sep 07 12:00:21.621486 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:21.621469    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 12:00:21.828896 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:21.828857    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74f97db495-g77z9" event={"ID":"0b1521a0-83c2-41d0-a62f-e4d6b80651e6","Type":"ContainerStarted","Data":"d5c5bc7df01fd8f52e795102f47e3da079c085c68672d9246450f3363e682bf1"}
Sep 07 12:00:24.843144 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:24.843099    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74f97db495-g77z9" event={"ID":"0b1521a0-83c2-41d0-a62f-e4d6b80651e6","Type":"ContainerStarted","Data":"3754947bea84b6e225b57fed34d61bd89e51a015bb1c5d6a09f6a7922f74bd6b"}
Sep 07 12:00:24.843628 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:24.843329    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:24.859917 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:24.859856    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74f97db495-g77z9" podStartSLOduration=2.153685515 podStartE2EDuration="4.859842858s" totalImagesPullingTime="2.706157343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:00:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:00:24.858482891 +0000 UTC m=+779.354372313" watchObservedRunningTime="2026-09-07 12:00:24.859842858 +0000 UTC m=+779.355732281"
Sep 07 12:00:35.814278 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:35.814243    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74f97db495-g77z9"]
Sep 07 12:00:35.814647 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:35.814451    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-74f97db495-g77z9" podUID="0b1521a0-83c2-41d0-a62f-e4d6b80651e6" containerName="manager" containerID="cri-o://3754947bea84b6e225b57fed34d61bd89e51a015bb1c5d6a09f6a7922f74bd6b" gracePeriod=30
Sep 07 12:00:35.816124 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:35.816101    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:36.060689 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.060662    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:36.186569 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.186534    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-webhook-cert\") pod \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\" (UID: \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\") "
Sep 07 12:00:36.186569 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.186572    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgdls\" (UniqueName: \"kubernetes.io/projected/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-kube-api-access-sgdls\") pod \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\" (UID: \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\") "
Sep 07 12:00:36.186826 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.186600    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-metrics-certs\") pod \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\" (UID: \"0b1521a0-83c2-41d0-a62f-e4d6b80651e6\") "
Sep 07 12:00:36.188855 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.188822    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0b1521a0-83c2-41d0-a62f-e4d6b80651e6" (UID: "0b1521a0-83c2-41d0-a62f-e4d6b80651e6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:00:36.188855 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.188846    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0b1521a0-83c2-41d0-a62f-e4d6b80651e6" (UID: "0b1521a0-83c2-41d0-a62f-e4d6b80651e6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 12:00:36.189044 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.188868    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-kube-api-access-sgdls" (OuterVolumeSpecName: "kube-api-access-sgdls") pod "0b1521a0-83c2-41d0-a62f-e4d6b80651e6" (UID: "0b1521a0-83c2-41d0-a62f-e4d6b80651e6"). InnerVolumeSpecName "kube-api-access-sgdls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:00:36.288091 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.288054    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-webhook-cert\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:00:36.288091 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.288087    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgdls\" (UniqueName: \"kubernetes.io/projected/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-kube-api-access-sgdls\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:00:36.288091 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.288097    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b1521a0-83c2-41d0-a62f-e4d6b80651e6-metrics-certs\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:00:36.891275 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.891242    2580 generic.go:356] "Generic (PLEG): container finished" podID="0b1521a0-83c2-41d0-a62f-e4d6b80651e6" containerID="3754947bea84b6e225b57fed34d61bd89e51a015bb1c5d6a09f6a7922f74bd6b" exitCode=0
Sep 07 12:00:36.891673 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.891302    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74f97db495-g77z9"
Sep 07 12:00:36.891673 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.891325    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74f97db495-g77z9" event={"ID":"0b1521a0-83c2-41d0-a62f-e4d6b80651e6","Type":"ContainerDied","Data":"3754947bea84b6e225b57fed34d61bd89e51a015bb1c5d6a09f6a7922f74bd6b"}
Sep 07 12:00:36.891673 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.891400    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74f97db495-g77z9" event={"ID":"0b1521a0-83c2-41d0-a62f-e4d6b80651e6","Type":"ContainerDied","Data":"d5c5bc7df01fd8f52e795102f47e3da079c085c68672d9246450f3363e682bf1"}
Sep 07 12:00:36.891673 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.891423    2580 scope.go:117] "RemoveContainer" containerID="3754947bea84b6e225b57fed34d61bd89e51a015bb1c5d6a09f6a7922f74bd6b"
Sep 07 12:00:36.900533 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.900516    2580 scope.go:117] "RemoveContainer" containerID="3754947bea84b6e225b57fed34d61bd89e51a015bb1c5d6a09f6a7922f74bd6b"
Sep 07 12:00:36.900816 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:00:36.900798    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3754947bea84b6e225b57fed34d61bd89e51a015bb1c5d6a09f6a7922f74bd6b\": container with ID starting with 3754947bea84b6e225b57fed34d61bd89e51a015bb1c5d6a09f6a7922f74bd6b not found: ID does not exist" containerID="3754947bea84b6e225b57fed34d61bd89e51a015bb1c5d6a09f6a7922f74bd6b"
Sep 07 12:00:36.900863 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.900825    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3754947bea84b6e225b57fed34d61bd89e51a015bb1c5d6a09f6a7922f74bd6b"} err="failed to get container status \"3754947bea84b6e225b57fed34d61bd89e51a015bb1c5d6a09f6a7922f74bd6b\": rpc error: code = NotFound desc = could not find container \"3754947bea84b6e225b57fed34d61bd89e51a015bb1c5d6a09f6a7922f74bd6b\": container with ID starting with 3754947bea84b6e225b57fed34d61bd89e51a015bb1c5d6a09f6a7922f74bd6b not found: ID does not exist"
Sep 07 12:00:36.913763 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.913709    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74f97db495-g77z9"]
Sep 07 12:00:36.915427 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:36.915404    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74f97db495-g77z9"]
Sep 07 12:00:38.009956 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:38.009914    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b1521a0-83c2-41d0-a62f-e4d6b80651e6" path="/var/lib/kubelet/pods/0b1521a0-83c2-41d0-a62f-e4d6b80651e6/volumes"
Sep 07 12:00:59.680193 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:59.680114    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-5kjj5"]
Sep 07 12:00:59.680674 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:59.680467    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b1521a0-83c2-41d0-a62f-e4d6b80651e6" containerName="manager"
Sep 07 12:00:59.680674 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:59.680482    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b1521a0-83c2-41d0-a62f-e4d6b80651e6" containerName="manager"
Sep 07 12:00:59.680674 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:59.680532    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b1521a0-83c2-41d0-a62f-e4d6b80651e6" containerName="manager"
Sep 07 12:00:59.687286 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:59.687262    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-5kjj5"
Sep 07 12:00:59.689929 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:59.689902    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-8rkvz\""
Sep 07 12:00:59.690055 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:59.689902    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 07 12:00:59.691939 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:59.691912    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eaded52-ec7d-4d87-af2f-cd8e7cfe2049-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-5kjj5\" (UID: \"8eaded52-ec7d-4d87-af2f-cd8e7cfe2049\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-5kjj5"
Sep 07 12:00:59.692121 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:59.692102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl9mh\" (UniqueName: \"kubernetes.io/projected/8eaded52-ec7d-4d87-af2f-cd8e7cfe2049-kube-api-access-wl9mh\") pod \"payload-processing-5b54b7fbb7-5kjj5\" (UID: \"8eaded52-ec7d-4d87-af2f-cd8e7cfe2049\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-5kjj5"
Sep 07 12:00:59.694087 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:59.694066    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-5kjj5"]
Sep 07 12:00:59.792978 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:59.792943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl9mh\" (UniqueName: \"kubernetes.io/projected/8eaded52-ec7d-4d87-af2f-cd8e7cfe2049-kube-api-access-wl9mh\") pod \"payload-processing-5b54b7fbb7-5kjj5\" (UID: \"8eaded52-ec7d-4d87-af2f-cd8e7cfe2049\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-5kjj5"
Sep 07 12:00:59.793139 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:59.793075    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eaded52-ec7d-4d87-af2f-cd8e7cfe2049-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-5kjj5\" (UID: \"8eaded52-ec7d-4d87-af2f-cd8e7cfe2049\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-5kjj5"
Sep 07 12:00:59.793802 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:59.793780    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8eaded52-ec7d-4d87-af2f-cd8e7cfe2049-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-5kjj5\" (UID: \"8eaded52-ec7d-4d87-af2f-cd8e7cfe2049\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-5kjj5"
Sep 07 12:00:59.801277 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:59.801249    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl9mh\" (UniqueName: \"kubernetes.io/projected/8eaded52-ec7d-4d87-af2f-cd8e7cfe2049-kube-api-access-wl9mh\") pod \"payload-processing-5b54b7fbb7-5kjj5\" (UID: \"8eaded52-ec7d-4d87-af2f-cd8e7cfe2049\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-5kjj5"
Sep 07 12:00:59.998019 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:00:59.997937    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-5kjj5"
Sep 07 12:01:00.274231 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:00.274186    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-5kjj5"]
Sep 07 12:01:00.275987 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:01:00.275952    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eaded52_ec7d_4d87_af2f_cd8e7cfe2049.slice/crio-fb51f6c7d81b663078d8af95ec97613de9e90565810256edf96312e16f4872d3 WatchSource:0}: Error finding container fb51f6c7d81b663078d8af95ec97613de9e90565810256edf96312e16f4872d3: Status 404 returned error can't find the container with id fb51f6c7d81b663078d8af95ec97613de9e90565810256edf96312e16f4872d3
Sep 07 12:01:00.991384 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:00.991352    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-5kjj5" event={"ID":"8eaded52-ec7d-4d87-af2f-cd8e7cfe2049","Type":"ContainerStarted","Data":"fb51f6c7d81b663078d8af95ec97613de9e90565810256edf96312e16f4872d3"}
Sep 07 12:01:04.009729 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:04.009702    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-5kjj5"
Sep 07 12:01:04.010184 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:04.009757    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-5kjj5" event={"ID":"8eaded52-ec7d-4d87-af2f-cd8e7cfe2049","Type":"ContainerStarted","Data":"4acf91c0648d13c2515089192f0557e1b375bdf519ca9d9885aeb765748b7e5a"}
Sep 07 12:01:04.056522 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:04.056463    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-5kjj5" podStartSLOduration=1.419695299 podStartE2EDuration="5.056440998s" totalImagesPullingTime="3.636745699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:00:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:01:04.054259888 +0000 UTC m=+818.550149313" watchObservedRunningTime="2026-09-07 12:01:04.056440998 +0000 UTC m=+818.552330422"
Sep 07 12:01:08.795641 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:08.795605    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6pgwj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.19:15021/healthz/ready\": read tcp 10.134.0.2:54226->10.134.0.19:15021: read: connection reset by peer" start-of-body=
Sep 07 12:01:08.796251 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:08.795671    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj" podUID="5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.19:15021/healthz/ready\": read tcp 10.134.0.2:54226->10.134.0.19:15021: read: connection reset by peer"
Sep 07 12:01:09.029021 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:09.028940    2580 generic.go:356] "Generic (PLEG): container finished" podID="5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952" containerID="5f16842afdb55056b6e8d779276a254c84642c963836f0ce6ee523450c55564a" exitCode=0
Sep 07 12:01:09.029175 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:09.029013    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj" event={"ID":"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952","Type":"ContainerDied","Data":"5f16842afdb55056b6e8d779276a254c84642c963836f0ce6ee523450c55564a"}
Sep 07 12:01:09.029448 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:09.029430    2580 scope.go:117] "RemoveContainer" containerID="5f16842afdb55056b6e8d779276a254c84642c963836f0ce6ee523450c55564a"
Sep 07 12:01:09.030422 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:09.030397    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:01:09.030482 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:09.030464    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:01:09.030528 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:09.030496    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:01:10.035513 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:10.035475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj" event={"ID":"5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952","Type":"ContainerStarted","Data":"410cbb17c9dd995b3835ea4c7e2eaca3ae182c054a0a9bc9364289d0553b361b"}
Sep 07 12:01:10.982883 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:10.982834    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 12:01:10.983135 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:10.983110    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6pgwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.19:15021/healthz/ready\": dial tcp 10.134.0.19:15021: connect: connection refused" start-of-body=
Sep 07 12:01:10.983249 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:10.983169    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj" podUID="5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.19:15021/healthz/ready\": dial tcp 10.134.0.19:15021: connect: connection refused"
Sep 07 12:01:11.983261 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:11.983188    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6pgwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.19:15021/healthz/ready\": dial tcp 10.134.0.19:15021: connect: connection refused" start-of-body=
Sep 07 12:01:11.983653 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:11.983292    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj" podUID="5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.19:15021/healthz/ready\": dial tcp 10.134.0.19:15021: connect: connection refused"
Sep 07 12:01:12.982947 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:12.982908    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6pgwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.19:15021/healthz/ready\": dial tcp 10.134.0.19:15021: connect: connection refused" start-of-body=
Sep 07 12:01:12.983106 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:12.982978    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj" podUID="5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.19:15021/healthz/ready\": dial tcp 10.134.0.19:15021: connect: connection refused"
Sep 07 12:01:13.983121 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:13.983080    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6pgwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.19:15021/healthz/ready\": dial tcp 10.134.0.19:15021: connect: connection refused" start-of-body=
Sep 07 12:01:13.983576 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:13.983140    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj" podUID="5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.19:15021/healthz/ready\": dial tcp 10.134.0.19:15021: connect: connection refused"
Sep 07 12:01:14.983544 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:14.983503    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6pgwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.19:15021/healthz/ready\": dial tcp 10.134.0.19:15021: connect: connection refused" start-of-body=
Sep 07 12:01:14.983959 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:14.983563    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj" podUID="5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.19:15021/healthz/ready\": dial tcp 10.134.0.19:15021: connect: connection refused"
Sep 07 12:01:15.013244 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:15.013190    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-5kjj5"
Sep 07 12:01:15.986666 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:15.986635    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 12:01:15.987042 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:15.986981    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 12:01:15.987742 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:15.987724    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6pgwj"
Sep 07 12:01:32.297217 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.297159    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"]
Sep 07 12:01:32.300565 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.300542    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.303032 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.303008    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 07 12:01:32.303951 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.303922    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 07 12:01:32.303951 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.303935    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 07 12:01:32.304120 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.303924    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-59vv8\""
Sep 07 12:01:32.311313 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.311290    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"]
Sep 07 12:01:32.379574 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.379537    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cc84c3c-5818-470f-8404-5492da0d92d6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.379771 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.379583    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cc84c3c-5818-470f-8404-5492da0d92d6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.379771 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.379609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cc84c3c-5818-470f-8404-5492da0d92d6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.379771 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.379664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2rzq\" (UniqueName: \"kubernetes.io/projected/6cc84c3c-5818-470f-8404-5492da0d92d6-kube-api-access-t2rzq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.379771 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.379700    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cc84c3c-5818-470f-8404-5492da0d92d6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.379771 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.379719    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cc84c3c-5818-470f-8404-5492da0d92d6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.480409 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.480362    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cc84c3c-5818-470f-8404-5492da0d92d6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.480592 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.480427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cc84c3c-5818-470f-8404-5492da0d92d6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.480592 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.480466    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cc84c3c-5818-470f-8404-5492da0d92d6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.480592 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.480496    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2rzq\" (UniqueName: \"kubernetes.io/projected/6cc84c3c-5818-470f-8404-5492da0d92d6-kube-api-access-t2rzq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.480755 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.480628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cc84c3c-5818-470f-8404-5492da0d92d6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.480755 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.480659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cc84c3c-5818-470f-8404-5492da0d92d6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.480865 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.480835    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6cc84c3c-5818-470f-8404-5492da0d92d6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.481306 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.481219    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6cc84c3c-5818-470f-8404-5492da0d92d6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.481306 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.481258    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cc84c3c-5818-470f-8404-5492da0d92d6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.483571 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.483508    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6cc84c3c-5818-470f-8404-5492da0d92d6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.483793 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.483773    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6cc84c3c-5818-470f-8404-5492da0d92d6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.488318 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.488295    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2rzq\" (UniqueName: \"kubernetes.io/projected/6cc84c3c-5818-470f-8404-5492da0d92d6-kube-api-access-t2rzq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4drnc\" (UID: \"6cc84c3c-5818-470f-8404-5492da0d92d6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.614295 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.614171    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:32.884158 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:32.884134    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"]
Sep 07 12:01:32.886068 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:01:32.886035    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc84c3c_5818_470f_8404_5492da0d92d6.slice/crio-18a845f0901381c91e5b49a4cccd805cd78dad86643877d73cd1c80112266f8a WatchSource:0}: Error finding container 18a845f0901381c91e5b49a4cccd805cd78dad86643877d73cd1c80112266f8a: Status 404 returned error can't find the container with id 18a845f0901381c91e5b49a4cccd805cd78dad86643877d73cd1c80112266f8a
Sep 07 12:01:33.135125 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:33.135039    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc" event={"ID":"6cc84c3c-5818-470f-8404-5492da0d92d6","Type":"ContainerStarted","Data":"18a845f0901381c91e5b49a4cccd805cd78dad86643877d73cd1c80112266f8a"}
Sep 07 12:01:34.140773 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:34.140739    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc" event={"ID":"6cc84c3c-5818-470f-8404-5492da0d92d6","Type":"ContainerStarted","Data":"1dccbb05c1ea9d53955a0bdb35b065b629c6fa598d507b8dde6be0efbe38c20d"}
Sep 07 12:01:34.157070 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:34.156912    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc" podStartSLOduration=1.123493126 podStartE2EDuration="2.156891532s" totalImagesPullingTime="1.033398406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:01:34.156089504 +0000 UTC m=+848.651978939" watchObservedRunningTime="2026-09-07 12:01:34.156891532 +0000 UTC m=+848.652780956"
Sep 07 12:01:42.615015 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:42.614971    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:42.615537 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:42.615086    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:42.627550 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:42.627520    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:43.191085 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:43.191052    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4drnc"
Sep 07 12:01:48.301523 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.301491    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"]
Sep 07 12:01:48.305469 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.305445    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.308110 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.308078    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 07 12:01:48.309974 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.309947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a97ca01-ebfc-4b5c-a963-95d99142d527-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.310094 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.309990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a97ca01-ebfc-4b5c-a963-95d99142d527-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.310094 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.310014    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a97ca01-ebfc-4b5c-a963-95d99142d527-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.310186 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.310108    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbbgj\" (UniqueName: \"kubernetes.io/projected/6a97ca01-ebfc-4b5c-a963-95d99142d527-kube-api-access-nbbgj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.310186 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.310165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a97ca01-ebfc-4b5c-a963-95d99142d527-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.310293 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.310221    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a97ca01-ebfc-4b5c-a963-95d99142d527-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.314900 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.314877    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"]
Sep 07 12:01:48.411082 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.411036    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a97ca01-ebfc-4b5c-a963-95d99142d527-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.411082 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.411078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a97ca01-ebfc-4b5c-a963-95d99142d527-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.411353 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.411139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbbgj\" (UniqueName: \"kubernetes.io/projected/6a97ca01-ebfc-4b5c-a963-95d99142d527-kube-api-access-nbbgj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.411353 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.411163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a97ca01-ebfc-4b5c-a963-95d99142d527-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.411353 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.411191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a97ca01-ebfc-4b5c-a963-95d99142d527-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.411353 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.411251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a97ca01-ebfc-4b5c-a963-95d99142d527-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.411567 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.411543    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a97ca01-ebfc-4b5c-a963-95d99142d527-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.411616 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.411560    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a97ca01-ebfc-4b5c-a963-95d99142d527-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.411670 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.411652    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a97ca01-ebfc-4b5c-a963-95d99142d527-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.413601 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.413579    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a97ca01-ebfc-4b5c-a963-95d99142d527-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.413782 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.413765    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a97ca01-ebfc-4b5c-a963-95d99142d527-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.419009 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.418984    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbbgj\" (UniqueName: \"kubernetes.io/projected/6a97ca01-ebfc-4b5c-a963-95d99142d527-kube-api-access-nbbgj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-gx45m\" (UID: \"6a97ca01-ebfc-4b5c-a963-95d99142d527\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.617217 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.617171    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:48.888784 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:48.888757    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"]
Sep 07 12:01:48.891007 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:01:48.890980    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a97ca01_ebfc_4b5c_a963_95d99142d527.slice/crio-dd1261c71677e85d06ebcd0de880cfc19b0d1fdebb8e889daf2eae93d113132f WatchSource:0}: Error finding container dd1261c71677e85d06ebcd0de880cfc19b0d1fdebb8e889daf2eae93d113132f: Status 404 returned error can't find the container with id dd1261c71677e85d06ebcd0de880cfc19b0d1fdebb8e889daf2eae93d113132f
Sep 07 12:01:49.211021 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:49.210925    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m" event={"ID":"6a97ca01-ebfc-4b5c-a963-95d99142d527","Type":"ContainerStarted","Data":"1a3af4c3b0e3b19a12b57691c0cf1be430df065f895f55b3c2dd1a73d7cc5e31"}
Sep 07 12:01:49.211021 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:49.210984    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m" event={"ID":"6a97ca01-ebfc-4b5c-a963-95d99142d527","Type":"ContainerStarted","Data":"dd1261c71677e85d06ebcd0de880cfc19b0d1fdebb8e889daf2eae93d113132f"}
Sep 07 12:01:49.232162 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:49.232105    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m" podStartSLOduration=1.043004021 podStartE2EDuration="1.232086869s" totalImagesPullingTime="189.082848ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:01:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:01:49.230623352 +0000 UTC m=+863.726512776" watchObservedRunningTime="2026-09-07 12:01:49.232086869 +0000 UTC m=+863.727976301"
Sep 07 12:01:58.617762 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:58.617721    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:58.618234 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:58.617797    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:58.630635 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:58.630605    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:01:59.272228 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:01:59.272181    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-gx45m"
Sep 07 12:03:48.311742 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.311655    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"]
Sep 07 12:03:48.315583 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.315557    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.318089 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.318068    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c7d751-openshift-default-dockercfg-jkdq5\""
Sep 07 12:03:48.326932 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.326902    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-workload-certs\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.327065 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.326946    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istiod-ca-cert\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.327065 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.326975    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2skb5\" (UniqueName: \"kubernetes.io/projected/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-kube-api-access-2skb5\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.327065 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.327009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-credential-socket\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.327065 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.327031    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-podinfo\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.327333 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.327089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-envoy\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.327333 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.327157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-data\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.327333 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.327223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-token\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.327333 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.327297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-workload-socket\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.327531 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.327394    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"]
Sep 07 12:03:48.427680 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.427638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-data\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.427680 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.427671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-token\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.427946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.427716    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-workload-socket\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.427946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.427735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-workload-certs\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.427946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.427753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istiod-ca-cert\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.427946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.427772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2skb5\" (UniqueName: \"kubernetes.io/projected/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-kube-api-access-2skb5\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.427946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.427804    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-credential-socket\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.427946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.427883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-podinfo\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.428312 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.427955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-envoy\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.428312 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.427988    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-data\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.428312 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.428301    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-workload-certs\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.428639 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.428602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-credential-socket\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.428747 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.428318    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-workload-socket\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.429086 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.429032    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istiod-ca-cert\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.438465 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.431730    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-podinfo\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.438465 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.435496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-envoy\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.438465 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.436519    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2skb5\" (UniqueName: \"kubernetes.io/projected/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-kube-api-access-2skb5\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.440566 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.440540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-token\") pod \"e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") " pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.628727 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.628686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:48.912681 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.912656    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"]
Sep 07 12:03:48.915372 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:03:48.915344    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b47525c_a9fb_45a3_be5b_42b7e36f22eb.slice/crio-c8c694a95bf1345be1860d68e6cd25e0d145649ee1a9e594183b8cc1a97def15 WatchSource:0}: Error finding container c8c694a95bf1345be1860d68e6cd25e0d145649ee1a9e594183b8cc1a97def15: Status 404 returned error can't find the container with id c8c694a95bf1345be1860d68e6cd25e0d145649ee1a9e594183b8cc1a97def15
Sep 07 12:03:48.917514 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.917483    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:03:48.917581 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.917551    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:03:48.917617 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:48.917580    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:03:49.741215 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:49.741168    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62" event={"ID":"6b47525c-a9fb-45a3-be5b-42b7e36f22eb","Type":"ContainerStarted","Data":"6cfd1e4a4ac74b944d6417420abb3eca3275962ba1f814483e3eefb03d672403"}
Sep 07 12:03:49.741643 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:49.741236    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62" event={"ID":"6b47525c-a9fb-45a3-be5b-42b7e36f22eb","Type":"ContainerStarted","Data":"c8c694a95bf1345be1860d68e6cd25e0d145649ee1a9e594183b8cc1a97def15"}
Sep 07 12:03:49.770318 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:49.770269    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62" podStartSLOduration=1.7702528210000001 podStartE2EDuration="1.770252821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:03:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:03:49.769434641 +0000 UTC m=+984.265324065" watchObservedRunningTime="2026-09-07 12:03:49.770252821 +0000 UTC m=+984.266142244"
Sep 07 12:03:50.629832 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:50.629796    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:50.634803 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:50.634775    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:50.746108 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:50.746079    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:50.747005 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:50.746985    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:03:53.525512 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.525480    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"]
Sep 07 12:03:53.529235 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.529215    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"
Sep 07 12:03:53.531740 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.531711    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-c7d751\""
Sep 07 12:03:53.539533 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.538248    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"]
Sep 07 12:03:53.545280 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.544087    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"
Sep 07 12:03:53.546597 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.546549    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"]
Sep 07 12:03:53.548066 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.546966    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-c7d751-dockercfg-gl9hk\""
Sep 07 12:03:53.549710 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.549685    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"]
Sep 07 12:03:53.570834 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.570802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88619833-32c5-4418-8925-91d78f45518a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv\" (UID: \"88619833-32c5-4418-8925-91d78f45518a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"
Sep 07 12:03:53.571002 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.570842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tt2d\" (UniqueName: \"kubernetes.io/projected/e5eb1e62-13b1-4973-aa34-ff4839fea413-kube-api-access-7tt2d\") pod \"payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h\" (UID: \"e5eb1e62-13b1-4973-aa34-ff4839fea413\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"
Sep 07 12:03:53.571002 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.570872    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5eb1e62-13b1-4973-aa34-ff4839fea413-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h\" (UID: \"e5eb1e62-13b1-4973-aa34-ff4839fea413\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"
Sep 07 12:03:53.571082 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.570997    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpvrz\" (UniqueName: \"kubernetes.io/projected/88619833-32c5-4418-8925-91d78f45518a-kube-api-access-xpvrz\") pod \"payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv\" (UID: \"88619833-32c5-4418-8925-91d78f45518a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"
Sep 07 12:03:53.672019 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.671978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88619833-32c5-4418-8925-91d78f45518a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv\" (UID: \"88619833-32c5-4418-8925-91d78f45518a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"
Sep 07 12:03:53.672223 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.672027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tt2d\" (UniqueName: \"kubernetes.io/projected/e5eb1e62-13b1-4973-aa34-ff4839fea413-kube-api-access-7tt2d\") pod \"payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h\" (UID: \"e5eb1e62-13b1-4973-aa34-ff4839fea413\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"
Sep 07 12:03:53.672223 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.672155    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5eb1e62-13b1-4973-aa34-ff4839fea413-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h\" (UID: \"e5eb1e62-13b1-4973-aa34-ff4839fea413\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"
Sep 07 12:03:53.672346 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.672259    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpvrz\" (UniqueName: \"kubernetes.io/projected/88619833-32c5-4418-8925-91d78f45518a-kube-api-access-xpvrz\") pod \"payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv\" (UID: \"88619833-32c5-4418-8925-91d78f45518a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"
Sep 07 12:03:53.672851 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.672832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5eb1e62-13b1-4973-aa34-ff4839fea413-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h\" (UID: \"e5eb1e62-13b1-4973-aa34-ff4839fea413\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"
Sep 07 12:03:53.672895 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.672837    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88619833-32c5-4418-8925-91d78f45518a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv\" (UID: \"88619833-32c5-4418-8925-91d78f45518a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"
Sep 07 12:03:53.680186 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.680135    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tt2d\" (UniqueName: \"kubernetes.io/projected/e5eb1e62-13b1-4973-aa34-ff4839fea413-kube-api-access-7tt2d\") pod \"payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h\" (UID: \"e5eb1e62-13b1-4973-aa34-ff4839fea413\") " pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"
Sep 07 12:03:53.680186 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.680170    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpvrz\" (UniqueName: \"kubernetes.io/projected/88619833-32c5-4418-8925-91d78f45518a-kube-api-access-xpvrz\") pod \"payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv\" (UID: \"88619833-32c5-4418-8925-91d78f45518a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"
Sep 07 12:03:53.847687 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.847598    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"
Sep 07 12:03:53.859718 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:53.859689    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"
Sep 07 12:03:54.119403 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:54.119291    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"]
Sep 07 12:03:54.122274 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:03:54.122245    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88619833_32c5_4418_8925_91d78f45518a.slice/crio-b10c220ae08451d4b32587d31ca6c448315ce63e24e0dfc7caff0f329467cdea WatchSource:0}: Error finding container b10c220ae08451d4b32587d31ca6c448315ce63e24e0dfc7caff0f329467cdea: Status 404 returned error can't find the container with id b10c220ae08451d4b32587d31ca6c448315ce63e24e0dfc7caff0f329467cdea
Sep 07 12:03:54.145138 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:54.145116    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"]
Sep 07 12:03:54.150249 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:03:54.150218    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5eb1e62_13b1_4973_aa34_ff4839fea413.slice/crio-65589822d77a0257c70e1b22e40f40f229b881d60d66bb0954e33b51db1f2a5e WatchSource:0}: Error finding container 65589822d77a0257c70e1b22e40f40f229b881d60d66bb0954e33b51db1f2a5e: Status 404 returned error can't find the container with id 65589822d77a0257c70e1b22e40f40f229b881d60d66bb0954e33b51db1f2a5e
Sep 07 12:03:54.765409 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:54.765371    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h" event={"ID":"e5eb1e62-13b1-4973-aa34-ff4839fea413","Type":"ContainerStarted","Data":"66e298d5dc84e36310f4acd484aad0d1af1b6d4c093a49826bcb137f09691f3b"}
Sep 07 12:03:54.765853 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:54.765431    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h" event={"ID":"e5eb1e62-13b1-4973-aa34-ff4839fea413","Type":"ContainerStarted","Data":"65589822d77a0257c70e1b22e40f40f229b881d60d66bb0954e33b51db1f2a5e"}
Sep 07 12:03:54.765853 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:54.765461    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"
Sep 07 12:03:54.767258 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:54.767233    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv" event={"ID":"88619833-32c5-4418-8925-91d78f45518a","Type":"ContainerStarted","Data":"9300d3a7be7e46773721a045ddf4fb67feaffe315d9bfd2aa946f5e45498416c"}
Sep 07 12:03:54.767380 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:54.767281    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv" event={"ID":"88619833-32c5-4418-8925-91d78f45518a","Type":"ContainerStarted","Data":"b10c220ae08451d4b32587d31ca6c448315ce63e24e0dfc7caff0f329467cdea"}
Sep 07 12:03:54.767380 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:54.767309    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"
Sep 07 12:03:54.783304 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:54.783136    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h" podStartSLOduration=1.783117935 podStartE2EDuration="1.783117935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:03:54.782484049 +0000 UTC m=+989.278373464" watchObservedRunningTime="2026-09-07 12:03:54.783117935 +0000 UTC m=+989.279007362"
Sep 07 12:03:54.798630 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:54.798577    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv" podStartSLOduration=1.7985627910000002 podStartE2EDuration="1.798562791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:03:54.796737278 +0000 UTC m=+989.292626702" watchObservedRunningTime="2026-09-07 12:03:54.798562791 +0000 UTC m=+989.294452214"
Sep 07 12:03:56.397085 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.397045    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"]
Sep 07 12:03:56.400935 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.400912    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.403357 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.403332    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d8796b1b-openshift-default-dockercfg-4q7mz\""
Sep 07 12:03:56.416957 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.416928    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"]
Sep 07 12:03:56.496997 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.496964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93b7b456-7891-4bde-8864-0f785202ed4f-istio-podinfo\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.497191 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.497017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-workload-socket\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.497191 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.497137    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-istio-envoy\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.497191 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.497177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93b7b456-7891-4bde-8864-0f785202ed4f-istiod-ca-cert\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.497367 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.497241    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-workload-certs\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.497367 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.497263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93b7b456-7891-4bde-8864-0f785202ed4f-istio-token\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.497367 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.497322    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-credential-socket\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.497367 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.497342    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-istio-data\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.497367 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.497356    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxszf\" (UniqueName: \"kubernetes.io/projected/93b7b456-7891-4bde-8864-0f785202ed4f-kube-api-access-rxszf\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.598267 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.598233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-istio-envoy\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.598267 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.598269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93b7b456-7891-4bde-8864-0f785202ed4f-istiod-ca-cert\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.598505 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.598286    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-workload-certs\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.598505 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.598399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93b7b456-7891-4bde-8864-0f785202ed4f-istio-token\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.598505 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.598477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-credential-socket\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.598662 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.598520    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-istio-data\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.598662 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.598548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxszf\" (UniqueName: \"kubernetes.io/projected/93b7b456-7891-4bde-8864-0f785202ed4f-kube-api-access-rxszf\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.598662 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.598592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93b7b456-7891-4bde-8864-0f785202ed4f-istio-podinfo\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.598810 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.598668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-workload-socket\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.598810 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.598740    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-workload-certs\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.598961 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.598925    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-istio-data\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.599034 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.598989    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93b7b456-7891-4bde-8864-0f785202ed4f-istiod-ca-cert\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.599089 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.599071    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-workload-socket\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.599185 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.599166    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-credential-socket\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.600878 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.600852    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-istio-envoy\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.601265 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.601245    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93b7b456-7891-4bde-8864-0f785202ed4f-istio-podinfo\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.606213 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.606176    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93b7b456-7891-4bde-8864-0f785202ed4f-istio-token\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.606382 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.606367    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxszf\" (UniqueName: \"kubernetes.io/projected/93b7b456-7891-4bde-8864-0f785202ed4f-kube-api-access-rxszf\") pod \"e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") " pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.715487 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.715384    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:56.992563 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.992532    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"]
Sep 07 12:03:56.994161 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:03:56.994131    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93b7b456_7891_4bde_8864_0f785202ed4f.slice/crio-e3e96f78d7391cacf699d20e4bf213e516293428c663be38cefade5578b53638 WatchSource:0}: Error finding container e3e96f78d7391cacf699d20e4bf213e516293428c663be38cefade5578b53638: Status 404 returned error can't find the container with id e3e96f78d7391cacf699d20e4bf213e516293428c663be38cefade5578b53638
Sep 07 12:03:56.996353 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.996323    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:03:56.996428 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.996385    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:03:56.996428 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:56.996414    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:03:57.783982 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:57.783945    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x" event={"ID":"93b7b456-7891-4bde-8864-0f785202ed4f","Type":"ContainerStarted","Data":"15a0cd6fb4c64e6c3b664362c2d8ccb7c1cdabd2b3ae97bd525e428071f9c27d"}
Sep 07 12:03:57.784382 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:57.784000    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x" event={"ID":"93b7b456-7891-4bde-8864-0f785202ed4f","Type":"ContainerStarted","Data":"e3e96f78d7391cacf699d20e4bf213e516293428c663be38cefade5578b53638"}
Sep 07 12:03:57.822320 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:57.822264    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x" podStartSLOduration=1.822244832 podStartE2EDuration="1.822244832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:03:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:03:57.821104375 +0000 UTC m=+992.316993797" watchObservedRunningTime="2026-09-07 12:03:57.822244832 +0000 UTC m=+992.318134257"
Sep 07 12:03:58.716365 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:58.716330    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:58.721319 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:58.721293    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:58.790570 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:58.790540    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:03:58.791747 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:03:58.791718    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:04:00.118818 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.118778    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"]
Sep 07 12:04:00.122951 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.122921    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.125628 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.125570    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-ebf845-openshift-default-dockercfg-zpdgd\""
Sep 07 12:04:00.130704 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.130666    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"]
Sep 07 12:04:00.229833 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.229786    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-workload-socket\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.230089 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.230069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-envoy\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.230304 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.230286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-token\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.230467 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.230452    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-data\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.230650 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.230620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-workload-certs\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.230776 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.230762    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bb2c73f-959f-488f-83e9-19a8a7537103-istiod-ca-cert\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.230926 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.230902    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndprw\" (UniqueName: \"kubernetes.io/projected/9bb2c73f-959f-488f-83e9-19a8a7537103-kube-api-access-ndprw\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.231057 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.231043    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-credential-socket\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.231231 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.231193    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-podinfo\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.332433 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.332392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-data\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.332623 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.332460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-workload-certs\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.332623 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.332485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bb2c73f-959f-488f-83e9-19a8a7537103-istiod-ca-cert\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.332623 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.332523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndprw\" (UniqueName: \"kubernetes.io/projected/9bb2c73f-959f-488f-83e9-19a8a7537103-kube-api-access-ndprw\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.332623 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.332564    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-credential-socket\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.332623 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.332607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-podinfo\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.332873 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.332638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-workload-socket\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.332873 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.332666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-envoy\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.332873 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.332731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-token\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.333002 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.332961    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-data\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.333333 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.333311    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-workload-socket\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.333439 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.333388    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bb2c73f-959f-488f-83e9-19a8a7537103-istiod-ca-cert\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.333694 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.333672    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-workload-certs\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.333879 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.333848    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-credential-socket\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.335646 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.335618    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-envoy\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.335895 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.335872    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-podinfo\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.340374 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.340330    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-token\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.343483 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.343458    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndprw\" (UniqueName: \"kubernetes.io/projected/9bb2c73f-959f-488f-83e9-19a8a7537103-kube-api-access-ndprw\") pod \"e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") " pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.444904 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.444790    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:00.736964 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.736829    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"]
Sep 07 12:04:00.740808 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:04:00.740783    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bb2c73f_959f_488f_83e9_19a8a7537103.slice/crio-5189d5daafd1811d1336169de6aef64a576bf6bdcb6a0663f53137037c067a23 WatchSource:0}: Error finding container 5189d5daafd1811d1336169de6aef64a576bf6bdcb6a0663f53137037c067a23: Status 404 returned error can't find the container with id 5189d5daafd1811d1336169de6aef64a576bf6bdcb6a0663f53137037c067a23
Sep 07 12:04:00.742998 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.742966    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:04:00.743083 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.743034    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:04:00.743083 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.743065    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:04:00.805706 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:00.805587    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg" event={"ID":"9bb2c73f-959f-488f-83e9-19a8a7537103","Type":"ContainerStarted","Data":"5189d5daafd1811d1336169de6aef64a576bf6bdcb6a0663f53137037c067a23"}
Sep 07 12:04:01.811522 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:01.811483    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg" event={"ID":"9bb2c73f-959f-488f-83e9-19a8a7537103","Type":"ContainerStarted","Data":"95097cf281ec4cc267d78d94597dbc679eccd6839e5d6374ed04c16d6df70883"}
Sep 07 12:04:01.832387 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:01.832325    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg" podStartSLOduration=1.832309248 podStartE2EDuration="1.832309248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:04:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:04:01.830066916 +0000 UTC m=+996.325956338" watchObservedRunningTime="2026-09-07 12:04:01.832309248 +0000 UTC m=+996.328198670"
Sep 07 12:04:02.445931 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:02.445889    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:02.457762 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:02.457501    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:02.816323 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:02.816238    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:02.817365 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:02.817347    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:05.570526 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:05.570491    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"]
Sep 07 12:04:05.573954 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:05.573930    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"
Sep 07 12:04:05.576228 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:05.576187    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-ebf845-dockercfg-gw6cn\""
Sep 07 12:04:05.576359 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:05.576325    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-ebf845\""
Sep 07 12:04:05.585357 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:05.585327    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"]
Sep 07 12:04:05.711448 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:05.711409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc362ffb-cd70-42d1-b46a-4dcc7ee724a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck\" (UID: \"fc362ffb-cd70-42d1-b46a-4dcc7ee724a8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"
Sep 07 12:04:05.711684 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:05.711660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2m4h\" (UniqueName: \"kubernetes.io/projected/fc362ffb-cd70-42d1-b46a-4dcc7ee724a8-kube-api-access-p2m4h\") pod \"payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck\" (UID: \"fc362ffb-cd70-42d1-b46a-4dcc7ee724a8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"
Sep 07 12:04:05.774132 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:05.774100    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"
Sep 07 12:04:05.774293 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:05.774191    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"
Sep 07 12:04:05.815145 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:05.813829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc362ffb-cd70-42d1-b46a-4dcc7ee724a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck\" (UID: \"fc362ffb-cd70-42d1-b46a-4dcc7ee724a8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"
Sep 07 12:04:05.815145 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:05.813879    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2m4h\" (UniqueName: \"kubernetes.io/projected/fc362ffb-cd70-42d1-b46a-4dcc7ee724a8-kube-api-access-p2m4h\") pod \"payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck\" (UID: \"fc362ffb-cd70-42d1-b46a-4dcc7ee724a8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"
Sep 07 12:04:05.815145 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:05.815121    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc362ffb-cd70-42d1-b46a-4dcc7ee724a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck\" (UID: \"fc362ffb-cd70-42d1-b46a-4dcc7ee724a8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"
Sep 07 12:04:05.829064 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:05.828910    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2m4h\" (UniqueName: \"kubernetes.io/projected/fc362ffb-cd70-42d1-b46a-4dcc7ee724a8-kube-api-access-p2m4h\") pod \"payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck\" (UID: \"fc362ffb-cd70-42d1-b46a-4dcc7ee724a8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"
Sep 07 12:04:05.886444 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:05.886406    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"
Sep 07 12:04:06.172221 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:06.171850    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"]
Sep 07 12:04:06.787380 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:06.787349    2580 patch_prober.go:28] interesting pod/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 07 12:04:06.787841 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:06.787805    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62" podUID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 07 12:04:06.841306 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:06.841265    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck" event={"ID":"fc362ffb-cd70-42d1-b46a-4dcc7ee724a8","Type":"ContainerStarted","Data":"5b52bb1977cbba6d27fa995c565069d0cdf0e0fde54b1058f9c763d7172f0192"}
Sep 07 12:04:06.841466 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:06.841322    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck" event={"ID":"fc362ffb-cd70-42d1-b46a-4dcc7ee724a8","Type":"ContainerStarted","Data":"8ca344aa23053fad46d598c9ec47ea159e5822bbd53ca1175c6ca071c3ab77b1"}
Sep 07 12:04:06.841466 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:06.841360    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"
Sep 07 12:04:06.859318 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:06.859247    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck" podStartSLOduration=1.859225893 podStartE2EDuration="1.859225893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:04:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:04:06.856126945 +0000 UTC m=+1001.352016369" watchObservedRunningTime="2026-09-07 12:04:06.859225893 +0000 UTC m=+1001.355115317"
Sep 07 12:04:08.854918 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:08.854820    2580 generic.go:356] "Generic (PLEG): container finished" podID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" containerID="6cfd1e4a4ac74b944d6417420abb3eca3275962ba1f814483e3eefb03d672403" exitCode=0
Sep 07 12:04:08.855611 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:08.855587    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62" event={"ID":"6b47525c-a9fb-45a3-be5b-42b7e36f22eb","Type":"ContainerDied","Data":"6cfd1e4a4ac74b944d6417420abb3eca3275962ba1f814483e3eefb03d672403"}
Sep 07 12:04:08.856247 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:08.856228    2580 scope.go:117] "RemoveContainer" containerID="6cfd1e4a4ac74b944d6417420abb3eca3275962ba1f814483e3eefb03d672403"
Sep 07 12:04:08.858089 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:08.858057    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:04:08.858311 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:08.858292    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:04:08.858472 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:08.858452    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:04:09.870489 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:09.870453    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62" event={"ID":"6b47525c-a9fb-45a3-be5b-42b7e36f22eb","Type":"ContainerStarted","Data":"164c041e4e423c6813b0a3f3a7122d99229e9da7a0c64bb9882890b202e75bc5"}
Sep 07 12:04:10.629264 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:10.629232    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:04:10.629517 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:10.629266    2580 patch_prober.go:28] interesting pod/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body=
Sep 07 12:04:10.629642 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:10.629623    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62" podUID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused"
Sep 07 12:04:11.629291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:11.629252    2580 patch_prober.go:28] interesting pod/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body=
Sep 07 12:04:11.629703 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:11.629314    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62" podUID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused"
Sep 07 12:04:12.630036 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:12.629995    2580 patch_prober.go:28] interesting pod/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body=
Sep 07 12:04:12.630454 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:12.630072    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62" podUID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused"
Sep 07 12:04:13.629727 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:13.629680    2580 patch_prober.go:28] interesting pod/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body=
Sep 07 12:04:13.629910 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:13.629760    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62" podUID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused"
Sep 07 12:04:14.629113 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:14.629071    2580 patch_prober.go:28] interesting pod/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body=
Sep 07 12:04:14.629517 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:14.629143    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62" podUID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused"
Sep 07 12:04:14.832409 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:14.832358    2580 patch_prober.go:28] interesting pod/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 07 12:04:14.832623 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:14.832431    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x" podUID="93b7b456-7891-4bde-8864-0f785202ed4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 07 12:04:15.633460 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:15.633424    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:04:15.633867 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:15.633848    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:04:15.634528 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:15.634509    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:04:16.915536 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:16.915501    2580 generic.go:356] "Generic (PLEG): container finished" podID="93b7b456-7891-4bde-8864-0f785202ed4f" containerID="15a0cd6fb4c64e6c3b664362c2d8ccb7c1cdabd2b3ae97bd525e428071f9c27d" exitCode=0
Sep 07 12:04:16.915992 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:16.915582    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x" event={"ID":"93b7b456-7891-4bde-8864-0f785202ed4f","Type":"ContainerDied","Data":"15a0cd6fb4c64e6c3b664362c2d8ccb7c1cdabd2b3ae97bd525e428071f9c27d"}
Sep 07 12:04:16.916130 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:16.916112    2580 scope.go:117] "RemoveContainer" containerID="15a0cd6fb4c64e6c3b664362c2d8ccb7c1cdabd2b3ae97bd525e428071f9c27d"
Sep 07 12:04:16.917109 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:16.917082    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:04:16.917228 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:16.917160    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:04:16.917291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:16.917235    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:04:17.850444 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:17.850404    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"
Sep 07 12:04:17.928237 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:17.928188    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x" event={"ID":"93b7b456-7891-4bde-8864-0f785202ed4f","Type":"ContainerStarted","Data":"1c2456bea4dd356624bacb892a6015f18f8e2ba1fa3da15bb222a478a2062024"}
Sep 07 12:04:18.715841 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:18.715741    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:04:18.716025 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:18.715996    2580 patch_prober.go:28] interesting pod/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 07 12:04:18.716100 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:18.716057    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x" podUID="93b7b456-7891-4bde-8864-0f785202ed4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 07 12:04:19.716488 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:19.716443    2580 patch_prober.go:28] interesting pod/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 07 12:04:19.716946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:19.716517    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x" podUID="93b7b456-7891-4bde-8864-0f785202ed4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 07 12:04:20.716053 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:20.716001    2580 patch_prober.go:28] interesting pod/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 07 12:04:20.716286 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:20.716062    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x" podUID="93b7b456-7891-4bde-8864-0f785202ed4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 07 12:04:21.716550 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:21.716512    2580 patch_prober.go:28] interesting pod/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 07 12:04:21.716956 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:21.716583    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x" podUID="93b7b456-7891-4bde-8864-0f785202ed4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 07 12:04:22.716624 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:22.716578    2580 patch_prober.go:28] interesting pod/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 07 12:04:22.717137 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:22.716646    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x" podUID="93b7b456-7891-4bde-8864-0f785202ed4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 07 12:04:22.969303 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:22.969188    2580 generic.go:356] "Generic (PLEG): container finished" podID="9bb2c73f-959f-488f-83e9-19a8a7537103" containerID="95097cf281ec4cc267d78d94597dbc679eccd6839e5d6374ed04c16d6df70883" exitCode=0
Sep 07 12:04:22.969303 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:22.969273    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg" event={"ID":"9bb2c73f-959f-488f-83e9-19a8a7537103","Type":"ContainerDied","Data":"95097cf281ec4cc267d78d94597dbc679eccd6839e5d6374ed04c16d6df70883"}
Sep 07 12:04:22.969788 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:22.969763    2580 scope.go:117] "RemoveContainer" containerID="95097cf281ec4cc267d78d94597dbc679eccd6839e5d6374ed04c16d6df70883"
Sep 07 12:04:22.971651 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:22.971605    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:04:22.971762 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:22.971711    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:04:22.971822 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:22.971768    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:04:23.719400 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:23.719364    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:04:23.719798 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:23.719781    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:04:23.720590 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:23.720572    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:04:23.978340 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:23.978243    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg" event={"ID":"9bb2c73f-959f-488f-83e9-19a8a7537103","Type":"ContainerStarted","Data":"98bed0f28db017bfa4ba90cf006e20bc286cb7e509878881872b4249b1ee373f"}
Sep 07 12:04:24.445616 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.445580    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:24.445845 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.445819    2580 patch_prober.go:28] interesting pod/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Sep 07 12:04:24.445930 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.445906    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg" podUID="9bb2c73f-959f-488f-83e9-19a8a7537103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Sep 07 12:04:24.631821 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.631788    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls"]
Sep 07 12:04:24.634320 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.634289    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls"
Sep 07 12:04:24.636824 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.636801    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:04:24.636934 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.636902    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:04:24.643005 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.642974    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls"]
Sep 07 12:04:24.726824 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.726728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64b26ada-c060-4505-9113-835d3a3b9b5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls\" (UID: \"64b26ada-c060-4505-9113-835d3a3b9b5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls"
Sep 07 12:04:24.828141 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.828099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64b26ada-c060-4505-9113-835d3a3b9b5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls\" (UID: \"64b26ada-c060-4505-9113-835d3a3b9b5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls"
Sep 07 12:04:24.828979 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.828953    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64b26ada-c060-4505-9113-835d3a3b9b5f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls\" (UID: \"64b26ada-c060-4505-9113-835d3a3b9b5f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls"
Sep 07 12:04:24.944401 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.944359    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"]
Sep 07 12:04:24.945221 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.945185    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls"
Sep 07 12:04:24.947933 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.947526    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"
Sep 07 12:04:24.952262 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.951097    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-af50c7-dockercfg-72lf2\""
Sep 07 12:04:24.952262 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.951410    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-af50c7\""
Sep 07 12:04:24.961652 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:24.961524    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"]
Sep 07 12:04:25.032636 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:25.032599    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34536cde-c715-4276-bf49-8a25d8f21f7f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj\" (UID: \"34536cde-c715-4276-bf49-8a25d8f21f7f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"
Sep 07 12:04:25.032867 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:25.032847    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctqbk\" (UniqueName: \"kubernetes.io/projected/34536cde-c715-4276-bf49-8a25d8f21f7f-kube-api-access-ctqbk\") pod \"payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj\" (UID: \"34536cde-c715-4276-bf49-8a25d8f21f7f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"
Sep 07 12:04:25.134233 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:25.134087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34536cde-c715-4276-bf49-8a25d8f21f7f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj\" (UID: \"34536cde-c715-4276-bf49-8a25d8f21f7f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"
Sep 07 12:04:25.134233 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:25.134163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctqbk\" (UniqueName: \"kubernetes.io/projected/34536cde-c715-4276-bf49-8a25d8f21f7f-kube-api-access-ctqbk\") pod \"payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj\" (UID: \"34536cde-c715-4276-bf49-8a25d8f21f7f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"
Sep 07 12:04:25.134914 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:25.134890    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34536cde-c715-4276-bf49-8a25d8f21f7f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj\" (UID: \"34536cde-c715-4276-bf49-8a25d8f21f7f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"
Sep 07 12:04:25.141821 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:25.141797    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctqbk\" (UniqueName: \"kubernetes.io/projected/34536cde-c715-4276-bf49-8a25d8f21f7f-kube-api-access-ctqbk\") pod \"payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj\" (UID: \"34536cde-c715-4276-bf49-8a25d8f21f7f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"
Sep 07 12:04:25.266263 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:25.266089    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls"]
Sep 07 12:04:25.277168 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:04:25.277130    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64b26ada_c060_4505_9113_835d3a3b9b5f.slice/crio-e183e3e0a619628ac975b3e086598e3e31603a3219405bcd4e6250d7e7084f4e WatchSource:0}: Error finding container e183e3e0a619628ac975b3e086598e3e31603a3219405bcd4e6250d7e7084f4e: Status 404 returned error can't find the container with id e183e3e0a619628ac975b3e086598e3e31603a3219405bcd4e6250d7e7084f4e
Sep 07 12:04:25.289697 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:25.289673    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"
Sep 07 12:04:25.445477 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:25.445436    2580 patch_prober.go:28] interesting pod/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Sep 07 12:04:25.445648 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:25.445497    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg" podUID="9bb2c73f-959f-488f-83e9-19a8a7537103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Sep 07 12:04:25.567372 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:25.567280    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"]
Sep 07 12:04:25.572805 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:04:25.572767    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34536cde_c715_4276_bf49_8a25d8f21f7f.slice/crio-dd3b5556370112541d16207fb31677272653a262e5a12a02a6571e67e3511b8d WatchSource:0}: Error finding container dd3b5556370112541d16207fb31677272653a262e5a12a02a6571e67e3511b8d: Status 404 returned error can't find the container with id dd3b5556370112541d16207fb31677272653a262e5a12a02a6571e67e3511b8d
Sep 07 12:04:25.992376 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:25.992335    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj" event={"ID":"34536cde-c715-4276-bf49-8a25d8f21f7f","Type":"ContainerStarted","Data":"da45089362dac8080fbc0bf6778f4b7eb55de24d186b5588d042e15b394c6f5b"}
Sep 07 12:04:25.992877 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:25.992397    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj" event={"ID":"34536cde-c715-4276-bf49-8a25d8f21f7f","Type":"ContainerStarted","Data":"dd3b5556370112541d16207fb31677272653a262e5a12a02a6571e67e3511b8d"}
Sep 07 12:04:25.992877 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:25.992427    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"
Sep 07 12:04:25.993545 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:25.993525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls" event={"ID":"64b26ada-c060-4505-9113-835d3a3b9b5f","Type":"ContainerStarted","Data":"e183e3e0a619628ac975b3e086598e3e31603a3219405bcd4e6250d7e7084f4e"}
Sep 07 12:04:26.009113 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:26.009060    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj" podStartSLOduration=2.009040995 podStartE2EDuration="2.009040995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:04:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:04:26.007939598 +0000 UTC m=+1020.503829020" watchObservedRunningTime="2026-09-07 12:04:26.009040995 +0000 UTC m=+1020.504930417"
Sep 07 12:04:26.446141 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:26.446098    2580 patch_prober.go:28] interesting pod/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Sep 07 12:04:26.446362 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:26.446175    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg" podUID="9bb2c73f-959f-488f-83e9-19a8a7537103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Sep 07 12:04:27.445533 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:27.445474    2580 patch_prober.go:28] interesting pod/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Sep 07 12:04:27.445993 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:27.445542    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg" podUID="9bb2c73f-959f-488f-83e9-19a8a7537103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Sep 07 12:04:28.445723 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:28.445616    2580 patch_prober.go:28] interesting pod/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Sep 07 12:04:28.445723 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:28.445681    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg" podUID="9bb2c73f-959f-488f-83e9-19a8a7537103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Sep 07 12:04:29.014801 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:29.014749    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls" event={"ID":"64b26ada-c060-4505-9113-835d3a3b9b5f","Type":"ContainerStarted","Data":"5bba5e1e3a80ad4c0ab578ba6342d6e91a7a07eb8f696dad03900478801d73f3"}
Sep 07 12:04:29.033432 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:29.033360    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls" podStartSLOduration=2.189898948 podStartE2EDuration="5.033336366s" totalImagesPullingTime="2.843437418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:04:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:04:29.030399977 +0000 UTC m=+1023.526289402" watchObservedRunningTime="2026-09-07 12:04:29.033336366 +0000 UTC m=+1023.529225791"
Sep 07 12:04:29.540952 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:29.540919    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:29.541390 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:29.541317    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:29.542211 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:29.542173    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:04:30.020804 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:30.020771    2580 generic.go:356] "Generic (PLEG): container finished" podID="64b26ada-c060-4505-9113-835d3a3b9b5f" containerID="5bba5e1e3a80ad4c0ab578ba6342d6e91a7a07eb8f696dad03900478801d73f3" exitCode=0
Sep 07 12:04:30.020988 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:30.020850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls" event={"ID":"64b26ada-c060-4505-9113-835d3a3b9b5f","Type":"ContainerDied","Data":"5bba5e1e3a80ad4c0ab578ba6342d6e91a7a07eb8f696dad03900478801d73f3"}
Sep 07 12:04:31.159399 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:31.159370    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls"
Sep 07 12:04:31.314463 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:31.314374    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64b26ada-c060-4505-9113-835d3a3b9b5f-maas-api-service-ca\") pod \"64b26ada-c060-4505-9113-835d3a3b9b5f\" (UID: \"64b26ada-c060-4505-9113-835d3a3b9b5f\") "
Sep 07 12:04:31.314729 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:31.314705    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64b26ada-c060-4505-9113-835d3a3b9b5f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "64b26ada-c060-4505-9113-835d3a3b9b5f" (UID: "64b26ada-c060-4505-9113-835d3a3b9b5f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:04:31.415993 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:31.415958    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64b26ada-c060-4505-9113-835d3a3b9b5f-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:04:32.031773 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:32.031749    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls"
Sep 07 12:04:32.031930 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:32.031748    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls" event={"ID":"64b26ada-c060-4505-9113-835d3a3b9b5f","Type":"ContainerDied","Data":"e183e3e0a619628ac975b3e086598e3e31603a3219405bcd4e6250d7e7084f4e"}
Sep 07 12:04:32.031930 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:32.031885    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e183e3e0a619628ac975b3e086598e3e31603a3219405bcd4e6250d7e7084f4e"
Sep 07 12:04:37.002289 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:37.002255    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"
Sep 07 12:04:41.547927 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:41.547882    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls"]
Sep 07 12:04:41.550211 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:41.550179    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d8796b1b-xgsls"]
Sep 07 12:04:41.674396 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:41.674364    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"]
Sep 07 12:04:41.674654 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:41.674607    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x" podUID="93b7b456-7891-4bde-8864-0f785202ed4f" containerName="istio-proxy" containerID="cri-o://1c2456bea4dd356624bacb892a6015f18f8e2ba1fa3da15bb222a478a2062024" gracePeriod=30
Sep 07 12:04:42.009651 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:42.009620    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64b26ada-c060-4505-9113-835d3a3b9b5f" path="/var/lib/kubelet/pods/64b26ada-c060-4505-9113-835d3a3b9b5f/volumes"
Sep 07 12:04:46.930567 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.930540    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:04:46.951483 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.951001    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93b7b456-7891-4bde-8864-0f785202ed4f-istio-podinfo\") pod \"93b7b456-7891-4bde-8864-0f785202ed4f\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") "
Sep 07 12:04:46.951483 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.951046    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93b7b456-7891-4bde-8864-0f785202ed4f-istio-token\") pod \"93b7b456-7891-4bde-8864-0f785202ed4f\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") "
Sep 07 12:04:46.951483 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.951168    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-istio-envoy\") pod \"93b7b456-7891-4bde-8864-0f785202ed4f\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") "
Sep 07 12:04:46.951483 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.951242    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-credential-socket\") pod \"93b7b456-7891-4bde-8864-0f785202ed4f\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") "
Sep 07 12:04:46.951483 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.951285    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93b7b456-7891-4bde-8864-0f785202ed4f-istiod-ca-cert\") pod \"93b7b456-7891-4bde-8864-0f785202ed4f\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") "
Sep 07 12:04:46.951483 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.951309    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-istio-data\") pod \"93b7b456-7891-4bde-8864-0f785202ed4f\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") "
Sep 07 12:04:46.951483 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.951332    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxszf\" (UniqueName: \"kubernetes.io/projected/93b7b456-7891-4bde-8864-0f785202ed4f-kube-api-access-rxszf\") pod \"93b7b456-7891-4bde-8864-0f785202ed4f\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") "
Sep 07 12:04:46.951483 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.951424    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-workload-certs\") pod \"93b7b456-7891-4bde-8864-0f785202ed4f\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") "
Sep 07 12:04:46.951483 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.951454    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-workload-socket\") pod \"93b7b456-7891-4bde-8864-0f785202ed4f\" (UID: \"93b7b456-7891-4bde-8864-0f785202ed4f\") "
Sep 07 12:04:46.952124 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.951859    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93b7b456-7891-4bde-8864-0f785202ed4f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "93b7b456-7891-4bde-8864-0f785202ed4f" (UID: "93b7b456-7891-4bde-8864-0f785202ed4f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:04:46.952124 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.951999    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "93b7b456-7891-4bde-8864-0f785202ed4f" (UID: "93b7b456-7891-4bde-8864-0f785202ed4f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:04:46.952276 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.952158    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-istio-data" (OuterVolumeSpecName: "istio-data") pod "93b7b456-7891-4bde-8864-0f785202ed4f" (UID: "93b7b456-7891-4bde-8864-0f785202ed4f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:04:46.952336 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.952271    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "93b7b456-7891-4bde-8864-0f785202ed4f" (UID: "93b7b456-7891-4bde-8864-0f785202ed4f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:04:46.952336 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.952286    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "93b7b456-7891-4bde-8864-0f785202ed4f" (UID: "93b7b456-7891-4bde-8864-0f785202ed4f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:04:46.953741 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.953674    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93b7b456-7891-4bde-8864-0f785202ed4f-istio-token" (OuterVolumeSpecName: "istio-token") pod "93b7b456-7891-4bde-8864-0f785202ed4f" (UID: "93b7b456-7891-4bde-8864-0f785202ed4f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:04:46.954149 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.954123    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93b7b456-7891-4bde-8864-0f785202ed4f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "93b7b456-7891-4bde-8864-0f785202ed4f" (UID: "93b7b456-7891-4bde-8864-0f785202ed4f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:04:46.954261 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.954129    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "93b7b456-7891-4bde-8864-0f785202ed4f" (UID: "93b7b456-7891-4bde-8864-0f785202ed4f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:04:46.954261 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:46.954155    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93b7b456-7891-4bde-8864-0f785202ed4f-kube-api-access-rxszf" (OuterVolumeSpecName: "kube-api-access-rxszf") pod "93b7b456-7891-4bde-8864-0f785202ed4f" (UID: "93b7b456-7891-4bde-8864-0f785202ed4f"). InnerVolumeSpecName "kube-api-access-rxszf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:04:47.052655 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.052614    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93b7b456-7891-4bde-8864-0f785202ed4f-istio-podinfo\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:04:47.052655 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.052646    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93b7b456-7891-4bde-8864-0f785202ed4f-istio-token\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:04:47.052655 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.052658    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-istio-envoy\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:04:47.052901 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.052669    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-credential-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:04:47.052901 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.052681    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93b7b456-7891-4bde-8864-0f785202ed4f-istiod-ca-cert\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:04:47.052901 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.052693    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-istio-data\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:04:47.052901 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.052703    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxszf\" (UniqueName: \"kubernetes.io/projected/93b7b456-7891-4bde-8864-0f785202ed4f-kube-api-access-rxszf\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:04:47.052901 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.052716    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-workload-certs\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:04:47.052901 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.052727    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93b7b456-7891-4bde-8864-0f785202ed4f-workload-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:04:47.107349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.107307    2580 generic.go:356] "Generic (PLEG): container finished" podID="93b7b456-7891-4bde-8864-0f785202ed4f" containerID="1c2456bea4dd356624bacb892a6015f18f8e2ba1fa3da15bb222a478a2062024" exitCode=0
Sep 07 12:04:47.107534 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.107451    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x" event={"ID":"93b7b456-7891-4bde-8864-0f785202ed4f","Type":"ContainerDied","Data":"1c2456bea4dd356624bacb892a6015f18f8e2ba1fa3da15bb222a478a2062024"}
Sep 07 12:04:47.107534 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.107504    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x" event={"ID":"93b7b456-7891-4bde-8864-0f785202ed4f","Type":"ContainerDied","Data":"e3e96f78d7391cacf699d20e4bf213e516293428c663be38cefade5578b53638"}
Sep 07 12:04:47.107534 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.107532    2580 scope.go:117] "RemoveContainer" containerID="1c2456bea4dd356624bacb892a6015f18f8e2ba1fa3da15bb222a478a2062024"
Sep 07 12:04:47.107723 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.107705    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"
Sep 07 12:04:47.128425 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.128365    2580 scope.go:117] "RemoveContainer" containerID="15a0cd6fb4c64e6c3b664362c2d8ccb7c1cdabd2b3ae97bd525e428071f9c27d"
Sep 07 12:04:47.146734 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.146713    2580 scope.go:117] "RemoveContainer" containerID="1c2456bea4dd356624bacb892a6015f18f8e2ba1fa3da15bb222a478a2062024"
Sep 07 12:04:47.147128 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:04:47.147101    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c2456bea4dd356624bacb892a6015f18f8e2ba1fa3da15bb222a478a2062024\": container with ID starting with 1c2456bea4dd356624bacb892a6015f18f8e2ba1fa3da15bb222a478a2062024 not found: ID does not exist" containerID="1c2456bea4dd356624bacb892a6015f18f8e2ba1fa3da15bb222a478a2062024"
Sep 07 12:04:47.147253 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.147137    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c2456bea4dd356624bacb892a6015f18f8e2ba1fa3da15bb222a478a2062024"} err="failed to get container status \"1c2456bea4dd356624bacb892a6015f18f8e2ba1fa3da15bb222a478a2062024\": rpc error: code = NotFound desc = could not find container \"1c2456bea4dd356624bacb892a6015f18f8e2ba1fa3da15bb222a478a2062024\": container with ID starting with 1c2456bea4dd356624bacb892a6015f18f8e2ba1fa3da15bb222a478a2062024 not found: ID does not exist"
Sep 07 12:04:47.147253 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.147157    2580 scope.go:117] "RemoveContainer" containerID="15a0cd6fb4c64e6c3b664362c2d8ccb7c1cdabd2b3ae97bd525e428071f9c27d"
Sep 07 12:04:47.147510 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:04:47.147484    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15a0cd6fb4c64e6c3b664362c2d8ccb7c1cdabd2b3ae97bd525e428071f9c27d\": container with ID starting with 15a0cd6fb4c64e6c3b664362c2d8ccb7c1cdabd2b3ae97bd525e428071f9c27d not found: ID does not exist" containerID="15a0cd6fb4c64e6c3b664362c2d8ccb7c1cdabd2b3ae97bd525e428071f9c27d"
Sep 07 12:04:47.147510 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.147515    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15a0cd6fb4c64e6c3b664362c2d8ccb7c1cdabd2b3ae97bd525e428071f9c27d"} err="failed to get container status \"15a0cd6fb4c64e6c3b664362c2d8ccb7c1cdabd2b3ae97bd525e428071f9c27d\": rpc error: code = NotFound desc = could not find container \"15a0cd6fb4c64e6c3b664362c2d8ccb7c1cdabd2b3ae97bd525e428071f9c27d\": container with ID starting with 15a0cd6fb4c64e6c3b664362c2d8ccb7c1cdabd2b3ae97bd525e428071f9c27d not found: ID does not exist"
Sep 07 12:04:47.150935 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.150910    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"]
Sep 07 12:04:47.154905 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:47.154883    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d8796b1b-openshift-default-5fc748bb58-kfs4x"]
Sep 07 12:04:48.009696 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.009663    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93b7b456-7891-4bde-8864-0f785202ed4f" path="/var/lib/kubelet/pods/93b7b456-7891-4bde-8864-0f785202ed4f/volumes"
Sep 07 12:04:48.777477 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.777440    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"]
Sep 07 12:04:48.777846 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.777832    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93b7b456-7891-4bde-8864-0f785202ed4f" containerName="istio-proxy"
Sep 07 12:04:48.777906 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.777848    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="93b7b456-7891-4bde-8864-0f785202ed4f" containerName="istio-proxy"
Sep 07 12:04:48.777906 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.777858    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64b26ada-c060-4505-9113-835d3a3b9b5f" containerName="revoke-keys"
Sep 07 12:04:48.777906 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.777864    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="64b26ada-c060-4505-9113-835d3a3b9b5f" containerName="revoke-keys"
Sep 07 12:04:48.777906 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.777874    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93b7b456-7891-4bde-8864-0f785202ed4f" containerName="istio-proxy"
Sep 07 12:04:48.777906 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.777879    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="93b7b456-7891-4bde-8864-0f785202ed4f" containerName="istio-proxy"
Sep 07 12:04:48.778052 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.777926    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="64b26ada-c060-4505-9113-835d3a3b9b5f" containerName="revoke-keys"
Sep 07 12:04:48.778052 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.777937    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="93b7b456-7891-4bde-8864-0f785202ed4f" containerName="istio-proxy"
Sep 07 12:04:48.778052 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.777943    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="93b7b456-7891-4bde-8864-0f785202ed4f" containerName="istio-proxy"
Sep 07 12:04:48.780939 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.780923    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"
Sep 07 12:04:48.783656 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.783629    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-efc88f2d-dockercfg-fw2qr\""
Sep 07 12:04:48.783775 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.783670    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-efc88f2d\""
Sep 07 12:04:48.789288 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.789258    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"]
Sep 07 12:04:48.871461 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.871268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef9fa466-1e04-44b8-9ad4-e8cc85a860b0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb\" (UID: \"ef9fa466-1e04-44b8-9ad4-e8cc85a860b0\") " pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"
Sep 07 12:04:48.871461 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.871313    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhq7n\" (UniqueName: \"kubernetes.io/projected/ef9fa466-1e04-44b8-9ad4-e8cc85a860b0-kube-api-access-dhq7n\") pod \"payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb\" (UID: \"ef9fa466-1e04-44b8-9ad4-e8cc85a860b0\") " pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"
Sep 07 12:04:48.972091 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.972048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef9fa466-1e04-44b8-9ad4-e8cc85a860b0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb\" (UID: \"ef9fa466-1e04-44b8-9ad4-e8cc85a860b0\") " pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"
Sep 07 12:04:48.972309 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.972106    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhq7n\" (UniqueName: \"kubernetes.io/projected/ef9fa466-1e04-44b8-9ad4-e8cc85a860b0-kube-api-access-dhq7n\") pod \"payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb\" (UID: \"ef9fa466-1e04-44b8-9ad4-e8cc85a860b0\") " pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"
Sep 07 12:04:48.972833 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.972811    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef9fa466-1e04-44b8-9ad4-e8cc85a860b0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb\" (UID: \"ef9fa466-1e04-44b8-9ad4-e8cc85a860b0\") " pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"
Sep 07 12:04:48.980730 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:48.980705    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhq7n\" (UniqueName: \"kubernetes.io/projected/ef9fa466-1e04-44b8-9ad4-e8cc85a860b0-kube-api-access-dhq7n\") pod \"payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb\" (UID: \"ef9fa466-1e04-44b8-9ad4-e8cc85a860b0\") " pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"
Sep 07 12:04:49.092287 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:49.092178    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"
Sep 07 12:04:49.363657 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:49.363631    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"]
Sep 07 12:04:49.366080 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:04:49.366052    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef9fa466_1e04_44b8_9ad4_e8cc85a860b0.slice/crio-72942dfd83c080e8317e91cd87c43f4cf6814b363353fc1032366d66fb0133c5 WatchSource:0}: Error finding container 72942dfd83c080e8317e91cd87c43f4cf6814b363353fc1032366d66fb0133c5: Status 404 returned error can't find the container with id 72942dfd83c080e8317e91cd87c43f4cf6814b363353fc1032366d66fb0133c5
Sep 07 12:04:50.126406 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:50.126367    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb" event={"ID":"ef9fa466-1e04-44b8-9ad4-e8cc85a860b0","Type":"ContainerStarted","Data":"90eb3a6fc6dc14da6157c109ce9e6f289355764bf440fa0cd5dcbf464a0a6f7a"}
Sep 07 12:04:50.126802 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:50.126428    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb" event={"ID":"ef9fa466-1e04-44b8-9ad4-e8cc85a860b0","Type":"ContainerStarted","Data":"72942dfd83c080e8317e91cd87c43f4cf6814b363353fc1032366d66fb0133c5"}
Sep 07 12:04:50.126802 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:50.126583    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"
Sep 07 12:04:50.145799 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:04:50.145728    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb" podStartSLOduration=2.145705429 podStartE2EDuration="2.145705429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:04:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:04:50.144942126 +0000 UTC m=+1044.640831550" watchObservedRunningTime="2026-09-07 12:04:50.145705429 +0000 UTC m=+1044.641594853"
Sep 07 12:05:01.134852 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:01.134821    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"
Sep 07 12:05:12.747646 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:12.747607    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96"]
Sep 07 12:05:12.752786 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:12.752754    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96"
Sep 07 12:05:12.755309 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:12.755284    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:05:12.755485 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:12.755307    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:05:12.755834 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:12.755804    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96"]
Sep 07 12:05:12.902867 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:12.902812    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/802b0cc5-0008-4033-be10-66c0820a6141-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96\" (UID: \"802b0cc5-0008-4033-be10-66c0820a6141\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96"
Sep 07 12:05:13.004001 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:13.003908    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/802b0cc5-0008-4033-be10-66c0820a6141-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96\" (UID: \"802b0cc5-0008-4033-be10-66c0820a6141\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96"
Sep 07 12:05:13.004703 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:13.004679    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/802b0cc5-0008-4033-be10-66c0820a6141-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96\" (UID: \"802b0cc5-0008-4033-be10-66c0820a6141\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96"
Sep 07 12:05:13.065212 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:13.065171    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96"
Sep 07 12:05:13.333228 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:13.333132    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96"]
Sep 07 12:05:13.336691 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:05:13.336655    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod802b0cc5_0008_4033_be10_66c0820a6141.slice/crio-4bc766c37c7fdc047b04433bc361232e50d2815706c76da6c5da1b97d9a8f351 WatchSource:0}: Error finding container 4bc766c37c7fdc047b04433bc361232e50d2815706c76da6c5da1b97d9a8f351: Status 404 returned error can't find the container with id 4bc766c37c7fdc047b04433bc361232e50d2815706c76da6c5da1b97d9a8f351
Sep 07 12:05:14.243514 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:14.243480    2580 generic.go:356] "Generic (PLEG): container finished" podID="802b0cc5-0008-4033-be10-66c0820a6141" containerID="7a1522db2404a5e874590de93ff7f84422492cf92541ef4aef73d61e6f7ed2df" exitCode=0
Sep 07 12:05:14.243864 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:14.243561    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96" event={"ID":"802b0cc5-0008-4033-be10-66c0820a6141","Type":"ContainerDied","Data":"7a1522db2404a5e874590de93ff7f84422492cf92541ef4aef73d61e6f7ed2df"}
Sep 07 12:05:14.243864 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:14.243609    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96" event={"ID":"802b0cc5-0008-4033-be10-66c0820a6141","Type":"ContainerStarted","Data":"4bc766c37c7fdc047b04433bc361232e50d2815706c76da6c5da1b97d9a8f351"}
Sep 07 12:05:15.385358 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:15.385332    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96"
Sep 07 12:05:15.528976 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:15.528882    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/802b0cc5-0008-4033-be10-66c0820a6141-maas-api-service-ca\") pod \"802b0cc5-0008-4033-be10-66c0820a6141\" (UID: \"802b0cc5-0008-4033-be10-66c0820a6141\") "
Sep 07 12:05:15.529298 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:15.529273    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/802b0cc5-0008-4033-be10-66c0820a6141-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "802b0cc5-0008-4033-be10-66c0820a6141" (UID: "802b0cc5-0008-4033-be10-66c0820a6141"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:05:15.630245 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:15.630186    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/802b0cc5-0008-4033-be10-66c0820a6141-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:05:16.255190 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:16.255159    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96"
Sep 07 12:05:16.255190 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:16.255173    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96" event={"ID":"802b0cc5-0008-4033-be10-66c0820a6141","Type":"ContainerDied","Data":"4bc766c37c7fdc047b04433bc361232e50d2815706c76da6c5da1b97d9a8f351"}
Sep 07 12:05:16.255413 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:16.255274    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4bc766c37c7fdc047b04433bc361232e50d2815706c76da6c5da1b97d9a8f351"
Sep 07 12:05:19.511476 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:19.511395    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"]
Sep 07 12:05:19.511975 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:19.511681    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb" podUID="ef9fa466-1e04-44b8-9ad4-e8cc85a860b0" containerName="payload-processing" containerID="cri-o://90eb3a6fc6dc14da6157c109ce9e6f289355764bf440fa0cd5dcbf464a0a6f7a" gracePeriod=30
Sep 07 12:05:19.673370 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:19.673334    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96"]
Sep 07 12:05:19.678739 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:19.678705    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-efc88f2d-xwf96"]
Sep 07 12:05:19.766113 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:19.765884    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"
Sep 07 12:05:19.867057 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:19.867023    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhq7n\" (UniqueName: \"kubernetes.io/projected/ef9fa466-1e04-44b8-9ad4-e8cc85a860b0-kube-api-access-dhq7n\") pod \"ef9fa466-1e04-44b8-9ad4-e8cc85a860b0\" (UID: \"ef9fa466-1e04-44b8-9ad4-e8cc85a860b0\") "
Sep 07 12:05:19.867282 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:19.867083    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef9fa466-1e04-44b8-9ad4-e8cc85a860b0-plugins-config-volume\") pod \"ef9fa466-1e04-44b8-9ad4-e8cc85a860b0\" (UID: \"ef9fa466-1e04-44b8-9ad4-e8cc85a860b0\") "
Sep 07 12:05:19.867407 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:19.867385    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef9fa466-1e04-44b8-9ad4-e8cc85a860b0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ef9fa466-1e04-44b8-9ad4-e8cc85a860b0" (UID: "ef9fa466-1e04-44b8-9ad4-e8cc85a860b0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:05:19.869057 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:19.869038    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef9fa466-1e04-44b8-9ad4-e8cc85a860b0-kube-api-access-dhq7n" (OuterVolumeSpecName: "kube-api-access-dhq7n") pod "ef9fa466-1e04-44b8-9ad4-e8cc85a860b0" (UID: "ef9fa466-1e04-44b8-9ad4-e8cc85a860b0"). InnerVolumeSpecName "kube-api-access-dhq7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:05:19.968443 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:19.968387    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhq7n\" (UniqueName: \"kubernetes.io/projected/ef9fa466-1e04-44b8-9ad4-e8cc85a860b0-kube-api-access-dhq7n\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:05:19.968443 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:19.968435    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef9fa466-1e04-44b8-9ad4-e8cc85a860b0-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:05:20.009686 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.009653    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="802b0cc5-0008-4033-be10-66c0820a6141" path="/var/lib/kubelet/pods/802b0cc5-0008-4033-be10-66c0820a6141/volumes"
Sep 07 12:05:20.275680 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.275578    2580 generic.go:356] "Generic (PLEG): container finished" podID="ef9fa466-1e04-44b8-9ad4-e8cc85a860b0" containerID="90eb3a6fc6dc14da6157c109ce9e6f289355764bf440fa0cd5dcbf464a0a6f7a" exitCode=0
Sep 07 12:05:20.275680 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.275633    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"
Sep 07 12:05:20.275680 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.275671    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb" event={"ID":"ef9fa466-1e04-44b8-9ad4-e8cc85a860b0","Type":"ContainerDied","Data":"90eb3a6fc6dc14da6157c109ce9e6f289355764bf440fa0cd5dcbf464a0a6f7a"}
Sep 07 12:05:20.275960 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.275726    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb" event={"ID":"ef9fa466-1e04-44b8-9ad4-e8cc85a860b0","Type":"ContainerDied","Data":"72942dfd83c080e8317e91cd87c43f4cf6814b363353fc1032366d66fb0133c5"}
Sep 07 12:05:20.275960 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.275753    2580 scope.go:117] "RemoveContainer" containerID="90eb3a6fc6dc14da6157c109ce9e6f289355764bf440fa0cd5dcbf464a0a6f7a"
Sep 07 12:05:20.284316 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.284289    2580 scope.go:117] "RemoveContainer" containerID="90eb3a6fc6dc14da6157c109ce9e6f289355764bf440fa0cd5dcbf464a0a6f7a"
Sep 07 12:05:20.284608 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:05:20.284583    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90eb3a6fc6dc14da6157c109ce9e6f289355764bf440fa0cd5dcbf464a0a6f7a\": container with ID starting with 90eb3a6fc6dc14da6157c109ce9e6f289355764bf440fa0cd5dcbf464a0a6f7a not found: ID does not exist" containerID="90eb3a6fc6dc14da6157c109ce9e6f289355764bf440fa0cd5dcbf464a0a6f7a"
Sep 07 12:05:20.284662 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.284613    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90eb3a6fc6dc14da6157c109ce9e6f289355764bf440fa0cd5dcbf464a0a6f7a"} err="failed to get container status \"90eb3a6fc6dc14da6157c109ce9e6f289355764bf440fa0cd5dcbf464a0a6f7a\": rpc error: code = NotFound desc = could not find container \"90eb3a6fc6dc14da6157c109ce9e6f289355764bf440fa0cd5dcbf464a0a6f7a\": container with ID starting with 90eb3a6fc6dc14da6157c109ce9e6f289355764bf440fa0cd5dcbf464a0a6f7a not found: ID does not exist"
Sep 07 12:05:20.292123 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.292090    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"]
Sep 07 12:05:20.295438 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.295408    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-efc88f2d-76c476c67d-zqqqb"]
Sep 07 12:05:20.469365 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.469328    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"]
Sep 07 12:05:20.469944 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.469908    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef9fa466-1e04-44b8-9ad4-e8cc85a860b0" containerName="payload-processing"
Sep 07 12:05:20.469944 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.469934    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef9fa466-1e04-44b8-9ad4-e8cc85a860b0" containerName="payload-processing"
Sep 07 12:05:20.470142 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.469952    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="802b0cc5-0008-4033-be10-66c0820a6141" containerName="revoke-keys"
Sep 07 12:05:20.470142 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.469961    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="802b0cc5-0008-4033-be10-66c0820a6141" containerName="revoke-keys"
Sep 07 12:05:20.470142 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.470034    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef9fa466-1e04-44b8-9ad4-e8cc85a860b0" containerName="payload-processing"
Sep 07 12:05:20.470142 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.470050    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="802b0cc5-0008-4033-be10-66c0820a6141" containerName="revoke-keys"
Sep 07 12:05:20.473130 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.473099    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.475693 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.475673    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c9822a85-openshift-default-dockercfg-lr459\""
Sep 07 12:05:20.492246 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.492181    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"]
Sep 07 12:05:20.573575 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.573474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-data\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.574136 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.574104    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-credential-socket\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.574286 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.574152    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-workload-certs\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.574286 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.574176    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-envoy\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.574286 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.574278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdlpj\" (UniqueName: \"kubernetes.io/projected/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-kube-api-access-hdlpj\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.574501 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.574423    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istiod-ca-cert\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.574556 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.574509    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-workload-socket\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.574556 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.574544    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-podinfo\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.574652 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.574568    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-token\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.675517 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.675474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-credential-socket\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.675517 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.675516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-workload-certs\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.675769 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.675534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-envoy\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.675769 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.675579    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdlpj\" (UniqueName: \"kubernetes.io/projected/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-kube-api-access-hdlpj\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.675769 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.675609    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istiod-ca-cert\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.675769 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.675672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-workload-socket\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.675769 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.675703    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-podinfo\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.675769 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.675728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-token\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.675769 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.675753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-data\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.676136 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.675975    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-workload-certs\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.676193 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.676157    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-workload-socket\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.676193 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.676182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-credential-socket\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.676306 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.676289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-data\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.676463 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.676446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istiod-ca-cert\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.678461 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.678438    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-envoy\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.678563 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.678547    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-podinfo\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.687932 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.687900    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdlpj\" (UniqueName: \"kubernetes.io/projected/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-kube-api-access-hdlpj\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.688043 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.687933    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-token\") pod \"e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") " pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:20.795321 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:20.795284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:21.068762 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:21.068734    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"]
Sep 07 12:05:21.071273 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:05:21.071247    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6741dff_46c8_47b6_b4f8_34bd8a3e349d.slice/crio-22a8b739e13979b9bcc5bb672ccb8e25575de253751eea9d2bb76eaaebd947a7 WatchSource:0}: Error finding container 22a8b739e13979b9bcc5bb672ccb8e25575de253751eea9d2bb76eaaebd947a7: Status 404 returned error can't find the container with id 22a8b739e13979b9bcc5bb672ccb8e25575de253751eea9d2bb76eaaebd947a7
Sep 07 12:05:21.073397 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:21.073367    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:05:21.073469 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:21.073435    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:05:21.073469 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:21.073463    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:05:21.283892 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:21.283850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk" event={"ID":"a6741dff-46c8-47b6-b4f8-34bd8a3e349d","Type":"ContainerStarted","Data":"4703a330df8afc6fa40e4b3375cda893bfc4cb0c2c8fa20575ac3f541b230682"}
Sep 07 12:05:21.284086 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:21.283919    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk" event={"ID":"a6741dff-46c8-47b6-b4f8-34bd8a3e349d","Type":"ContainerStarted","Data":"22a8b739e13979b9bcc5bb672ccb8e25575de253751eea9d2bb76eaaebd947a7"}
Sep 07 12:05:21.305337 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:21.305272    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk" podStartSLOduration=1.305255316 podStartE2EDuration="1.305255316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:05:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:05:21.30400647 +0000 UTC m=+1075.799895893" watchObservedRunningTime="2026-09-07 12:05:21.305255316 +0000 UTC m=+1075.801144738"
Sep 07 12:05:21.796414 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:21.796375    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:22.011398 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:22.011366    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef9fa466-1e04-44b8-9ad4-e8cc85a860b0" path="/var/lib/kubelet/pods/ef9fa466-1e04-44b8-9ad4-e8cc85a860b0/volumes"
Sep 07 12:05:22.801564 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:22.801533    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:23.297117 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:23.297083    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:23.298081 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:23.298060    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:24.552049 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:24.552012    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"]
Sep 07 12:05:24.554524 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:24.554507    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"
Sep 07 12:05:24.557415 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:24.557391    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-c9822a85\""
Sep 07 12:05:24.557547 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:24.557397    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-c9822a85-dockercfg-hrt5b\""
Sep 07 12:05:24.564069 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:24.564040    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"]
Sep 07 12:05:24.716470 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:24.716425    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a630b7d-2458-48c4-b058-06873166fa26-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc\" (UID: \"6a630b7d-2458-48c4-b058-06873166fa26\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"
Sep 07 12:05:24.716681 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:24.716579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb9dx\" (UniqueName: \"kubernetes.io/projected/6a630b7d-2458-48c4-b058-06873166fa26-kube-api-access-gb9dx\") pod \"payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc\" (UID: \"6a630b7d-2458-48c4-b058-06873166fa26\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"
Sep 07 12:05:24.817989 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:24.817894    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a630b7d-2458-48c4-b058-06873166fa26-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc\" (UID: \"6a630b7d-2458-48c4-b058-06873166fa26\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"
Sep 07 12:05:24.818144 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:24.817998    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb9dx\" (UniqueName: \"kubernetes.io/projected/6a630b7d-2458-48c4-b058-06873166fa26-kube-api-access-gb9dx\") pod \"payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc\" (UID: \"6a630b7d-2458-48c4-b058-06873166fa26\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"
Sep 07 12:05:24.818739 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:24.818718    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a630b7d-2458-48c4-b058-06873166fa26-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc\" (UID: \"6a630b7d-2458-48c4-b058-06873166fa26\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"
Sep 07 12:05:24.825636 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:24.825608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb9dx\" (UniqueName: \"kubernetes.io/projected/6a630b7d-2458-48c4-b058-06873166fa26-kube-api-access-gb9dx\") pod \"payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc\" (UID: \"6a630b7d-2458-48c4-b058-06873166fa26\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"
Sep 07 12:05:24.866769 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:24.866733    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"
Sep 07 12:05:25.137467 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:25.137284    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"]
Sep 07 12:05:25.139598 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:05:25.139569    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a630b7d_2458_48c4_b058_06873166fa26.slice/crio-d7cdffd50eddd0b91782e6653cac1930eb5b069416ec6d0191fa1e33e726ed4b WatchSource:0}: Error finding container d7cdffd50eddd0b91782e6653cac1930eb5b069416ec6d0191fa1e33e726ed4b: Status 404 returned error can't find the container with id d7cdffd50eddd0b91782e6653cac1930eb5b069416ec6d0191fa1e33e726ed4b
Sep 07 12:05:25.309634 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:25.309598    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc" event={"ID":"6a630b7d-2458-48c4-b058-06873166fa26","Type":"ContainerStarted","Data":"6d2cca5ad1b4ac6d18428bc2f1c5deab6036324f035ee75aa9a1288f8f2210b9"}
Sep 07 12:05:25.309845 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:25.309660    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc" event={"ID":"6a630b7d-2458-48c4-b058-06873166fa26","Type":"ContainerStarted","Data":"d7cdffd50eddd0b91782e6653cac1930eb5b069416ec6d0191fa1e33e726ed4b"}
Sep 07 12:05:25.309909 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:25.309851    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"
Sep 07 12:05:25.326466 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:25.326411    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc" podStartSLOduration=1.326395661 podStartE2EDuration="1.326395661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:05:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:05:25.324814045 +0000 UTC m=+1079.820703469" watchObservedRunningTime="2026-09-07 12:05:25.326395661 +0000 UTC m=+1079.822285084"
Sep 07 12:05:36.317695 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:36.317663    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"
Sep 07 12:05:43.419253 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:43.419193    2580 generic.go:356] "Generic (PLEG): container finished" podID="a6741dff-46c8-47b6-b4f8-34bd8a3e349d" containerID="4703a330df8afc6fa40e4b3375cda893bfc4cb0c2c8fa20575ac3f541b230682" exitCode=0
Sep 07 12:05:43.419697 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:43.419242    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk" event={"ID":"a6741dff-46c8-47b6-b4f8-34bd8a3e349d","Type":"ContainerDied","Data":"4703a330df8afc6fa40e4b3375cda893bfc4cb0c2c8fa20575ac3f541b230682"}
Sep 07 12:05:43.419697 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:43.419687    2580 scope.go:117] "RemoveContainer" containerID="4703a330df8afc6fa40e4b3375cda893bfc4cb0c2c8fa20575ac3f541b230682"
Sep 07 12:05:43.420845 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:43.420821    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:05:43.420908 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:43.420894    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:05:43.420950 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:43.420929    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:05:44.425987 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:44.425954    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk" event={"ID":"a6741dff-46c8-47b6-b4f8-34bd8a3e349d","Type":"ContainerStarted","Data":"5309b0315f6fc735c08302593c664b8420000a7291f61e3263ad7e7232723227"}
Sep 07 12:05:44.795756 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:44.795653    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:44.795932 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:44.795848    2580 patch_prober.go:28] interesting pod/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 07 12:05:44.795932 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:44.795897    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk" podUID="a6741dff-46c8-47b6-b4f8-34bd8a3e349d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 07 12:05:45.400779 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:45.400737    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk"]
Sep 07 12:05:45.404871 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:45.404828    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk"
Sep 07 12:05:45.411445 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:45.409572    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:05:45.411445 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:45.409849    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:05:45.411445 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:45.411376    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk"]
Sep 07 12:05:45.521040 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:45.520998    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58c3f85d-8417-4b47-afb8-a603c176018f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-ebf845-wphsk\" (UID: \"58c3f85d-8417-4b47-afb8-a603c176018f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk"
Sep 07 12:05:45.622127 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:45.622085    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58c3f85d-8417-4b47-afb8-a603c176018f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-ebf845-wphsk\" (UID: \"58c3f85d-8417-4b47-afb8-a603c176018f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk"
Sep 07 12:05:45.622909 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:45.622879    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58c3f85d-8417-4b47-afb8-a603c176018f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-ebf845-wphsk\" (UID: \"58c3f85d-8417-4b47-afb8-a603c176018f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk"
Sep 07 12:05:45.722575 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:45.722474    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk"
Sep 07 12:05:45.796405 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:45.796372    2580 patch_prober.go:28] interesting pod/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 07 12:05:45.796575 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:45.796428    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk" podUID="a6741dff-46c8-47b6-b4f8-34bd8a3e349d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 07 12:05:46.026791 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:46.026765    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk"]
Sep 07 12:05:46.028878 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:05:46.028848    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58c3f85d_8417_4b47_afb8_a603c176018f.slice/crio-98dd277bdfa938bdd463c1c54f359b6257d4924eb5db3645a6685fb7873ab404 WatchSource:0}: Error finding container 98dd277bdfa938bdd463c1c54f359b6257d4924eb5db3645a6685fb7873ab404: Status 404 returned error can't find the container with id 98dd277bdfa938bdd463c1c54f359b6257d4924eb5db3645a6685fb7873ab404
Sep 07 12:05:46.439952 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:46.439915    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk" event={"ID":"58c3f85d-8417-4b47-afb8-a603c176018f","Type":"ContainerStarted","Data":"cc0897b76854629d99d7a7bc1b2f24ba658ba8abb453445a8bad49765708abe7"}
Sep 07 12:05:46.440134 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:46.439977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk" event={"ID":"58c3f85d-8417-4b47-afb8-a603c176018f","Type":"ContainerStarted","Data":"98dd277bdfa938bdd463c1c54f359b6257d4924eb5db3645a6685fb7873ab404"}
Sep 07 12:05:46.454857 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:46.454785    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk" podStartSLOduration=1.454763367 podStartE2EDuration="1.454763367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:05:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:05:46.453662139 +0000 UTC m=+1100.949551563" watchObservedRunningTime="2026-09-07 12:05:46.454763367 +0000 UTC m=+1100.950652792"
Sep 07 12:05:46.796050 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:46.796012    2580 patch_prober.go:28] interesting pod/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 07 12:05:46.796478 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:46.796068    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk" podUID="a6741dff-46c8-47b6-b4f8-34bd8a3e349d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 07 12:05:47.446687 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:47.446643    2580 generic.go:356] "Generic (PLEG): container finished" podID="58c3f85d-8417-4b47-afb8-a603c176018f" containerID="cc0897b76854629d99d7a7bc1b2f24ba658ba8abb453445a8bad49765708abe7" exitCode=0
Sep 07 12:05:47.446896 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:47.446725    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk" event={"ID":"58c3f85d-8417-4b47-afb8-a603c176018f","Type":"ContainerDied","Data":"cc0897b76854629d99d7a7bc1b2f24ba658ba8abb453445a8bad49765708abe7"}
Sep 07 12:05:47.795909 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:47.795817    2580 patch_prober.go:28] interesting pod/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 07 12:05:47.795909 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:47.795883    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk" podUID="a6741dff-46c8-47b6-b4f8-34bd8a3e349d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 07 12:05:48.298941 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:48.298900    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf"]
Sep 07 12:05:48.301449 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:48.301431    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf"
Sep 07 12:05:48.309325 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:48.309297    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf"]
Sep 07 12:05:48.474812 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:48.474766    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab214e9-cf9d-447b-999e-bb447f030cbf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf\" (UID: \"2ab214e9-cf9d-447b-999e-bb447f030cbf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf"
Sep 07 12:05:48.579918 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:48.579816    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab214e9-cf9d-447b-999e-bb447f030cbf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf\" (UID: \"2ab214e9-cf9d-447b-999e-bb447f030cbf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf"
Sep 07 12:05:48.580998 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:48.580969    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab214e9-cf9d-447b-999e-bb447f030cbf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf\" (UID: \"2ab214e9-cf9d-447b-999e-bb447f030cbf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf"
Sep 07 12:05:48.613328 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:48.613300    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf"
Sep 07 12:05:48.641980 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:48.641950    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk"
Sep 07 12:05:48.783259 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:48.783191    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58c3f85d-8417-4b47-afb8-a603c176018f-maas-api-service-ca\") pod \"58c3f85d-8417-4b47-afb8-a603c176018f\" (UID: \"58c3f85d-8417-4b47-afb8-a603c176018f\") "
Sep 07 12:05:48.783551 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:48.783530    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58c3f85d-8417-4b47-afb8-a603c176018f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "58c3f85d-8417-4b47-afb8-a603c176018f" (UID: "58c3f85d-8417-4b47-afb8-a603c176018f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:05:48.796437 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:48.796391    2580 patch_prober.go:28] interesting pod/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 07 12:05:48.796597 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:48.796459    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk" podUID="a6741dff-46c8-47b6-b4f8-34bd8a3e349d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 07 12:05:48.884370 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:48.884333    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58c3f85d-8417-4b47-afb8-a603c176018f-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:05:48.929475 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:48.929396    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf"]
Sep 07 12:05:49.462967 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:49.462871    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk"
Sep 07 12:05:49.462967 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:49.462912    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk" event={"ID":"58c3f85d-8417-4b47-afb8-a603c176018f","Type":"ContainerDied","Data":"98dd277bdfa938bdd463c1c54f359b6257d4924eb5db3645a6685fb7873ab404"}
Sep 07 12:05:49.463508 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:49.462969    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="98dd277bdfa938bdd463c1c54f359b6257d4924eb5db3645a6685fb7873ab404"
Sep 07 12:05:49.465428 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:49.465381    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf" event={"ID":"2ab214e9-cf9d-447b-999e-bb447f030cbf","Type":"ContainerStarted","Data":"0ff232ed2d4b5c37b01d34f5df5d89a5245c9f4b51a3ddb1014d8583b0cebd91"}
Sep 07 12:05:49.465552 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:49.465458    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf" event={"ID":"2ab214e9-cf9d-447b-999e-bb447f030cbf","Type":"ContainerStarted","Data":"808ab32f298c2c491bf7061eed6461624752a627a80a48a7bd85b89623e472f6"}
Sep 07 12:05:49.483119 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:49.483065    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf" podStartSLOduration=1.483048258 podStartE2EDuration="1.483048258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:05:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:05:49.481234228 +0000 UTC m=+1103.977123651" watchObservedRunningTime="2026-09-07 12:05:49.483048258 +0000 UTC m=+1103.978937715"
Sep 07 12:05:50.031642 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:50.031603    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:50.032033 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:50.032002    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:50.033174 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:50.033151    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:05:50.471818 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:50.471781    2580 generic.go:356] "Generic (PLEG): container finished" podID="2ab214e9-cf9d-447b-999e-bb447f030cbf" containerID="0ff232ed2d4b5c37b01d34f5df5d89a5245c9f4b51a3ddb1014d8583b0cebd91" exitCode=0
Sep 07 12:05:50.472307 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:50.471870    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf" event={"ID":"2ab214e9-cf9d-447b-999e-bb447f030cbf","Type":"ContainerDied","Data":"0ff232ed2d4b5c37b01d34f5df5d89a5245c9f4b51a3ddb1014d8583b0cebd91"}
Sep 07 12:05:51.608634 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:51.608607    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf"
Sep 07 12:05:51.711906 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:51.711867    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab214e9-cf9d-447b-999e-bb447f030cbf-maas-api-service-ca\") pod \"2ab214e9-cf9d-447b-999e-bb447f030cbf\" (UID: \"2ab214e9-cf9d-447b-999e-bb447f030cbf\") "
Sep 07 12:05:51.712273 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:51.712251    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ab214e9-cf9d-447b-999e-bb447f030cbf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2ab214e9-cf9d-447b-999e-bb447f030cbf" (UID: "2ab214e9-cf9d-447b-999e-bb447f030cbf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:05:51.812644 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:51.812556    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab214e9-cf9d-447b-999e-bb447f030cbf-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:05:52.482462 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:52.482428    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf" event={"ID":"2ab214e9-cf9d-447b-999e-bb447f030cbf","Type":"ContainerDied","Data":"808ab32f298c2c491bf7061eed6461624752a627a80a48a7bd85b89623e472f6"}
Sep 07 12:05:52.482462 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:52.482455    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf"
Sep 07 12:05:52.482668 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:52.482477    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="808ab32f298c2c491bf7061eed6461624752a627a80a48a7bd85b89623e472f6"
Sep 07 12:05:54.399824 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:54.399789    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"]
Sep 07 12:05:54.400330 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:54.400101    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc" podUID="6a630b7d-2458-48c4-b058-06873166fa26" containerName="payload-pre-processing" containerID="cri-o://6d2cca5ad1b4ac6d18428bc2f1c5deab6036324f035ee75aa9a1288f8f2210b9" gracePeriod=30
Sep 07 12:05:54.555853 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:54.555812    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf"]
Sep 07 12:05:54.558735 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:54.558705    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9822a85-6vcjf"]
Sep 07 12:05:54.659237 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:54.659114    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"
Sep 07 12:05:54.841266 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:54.841193    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gb9dx\" (UniqueName: \"kubernetes.io/projected/6a630b7d-2458-48c4-b058-06873166fa26-kube-api-access-gb9dx\") pod \"6a630b7d-2458-48c4-b058-06873166fa26\" (UID: \"6a630b7d-2458-48c4-b058-06873166fa26\") "
Sep 07 12:05:54.841440 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:54.841289    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a630b7d-2458-48c4-b058-06873166fa26-plugins-config-volume\") pod \"6a630b7d-2458-48c4-b058-06873166fa26\" (UID: \"6a630b7d-2458-48c4-b058-06873166fa26\") "
Sep 07 12:05:54.841717 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:54.841697    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a630b7d-2458-48c4-b058-06873166fa26-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6a630b7d-2458-48c4-b058-06873166fa26" (UID: "6a630b7d-2458-48c4-b058-06873166fa26"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:05:54.843360 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:54.843340    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a630b7d-2458-48c4-b058-06873166fa26-kube-api-access-gb9dx" (OuterVolumeSpecName: "kube-api-access-gb9dx") pod "6a630b7d-2458-48c4-b058-06873166fa26" (UID: "6a630b7d-2458-48c4-b058-06873166fa26"). InnerVolumeSpecName "kube-api-access-gb9dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:05:54.942427 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:54.942333    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gb9dx\" (UniqueName: \"kubernetes.io/projected/6a630b7d-2458-48c4-b058-06873166fa26-kube-api-access-gb9dx\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:05:54.942427 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:54.942365    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a630b7d-2458-48c4-b058-06873166fa26-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:05:55.499809 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:55.499755    2580 generic.go:356] "Generic (PLEG): container finished" podID="6a630b7d-2458-48c4-b058-06873166fa26" containerID="6d2cca5ad1b4ac6d18428bc2f1c5deab6036324f035ee75aa9a1288f8f2210b9" exitCode=0
Sep 07 12:05:55.500325 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:55.500003    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc" event={"ID":"6a630b7d-2458-48c4-b058-06873166fa26","Type":"ContainerDied","Data":"6d2cca5ad1b4ac6d18428bc2f1c5deab6036324f035ee75aa9a1288f8f2210b9"}
Sep 07 12:05:55.500325 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:55.500014    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"
Sep 07 12:05:55.500325 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:55.500088    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc" event={"ID":"6a630b7d-2458-48c4-b058-06873166fa26","Type":"ContainerDied","Data":"d7cdffd50eddd0b91782e6653cac1930eb5b069416ec6d0191fa1e33e726ed4b"}
Sep 07 12:05:55.500325 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:55.500119    2580 scope.go:117] "RemoveContainer" containerID="6d2cca5ad1b4ac6d18428bc2f1c5deab6036324f035ee75aa9a1288f8f2210b9"
Sep 07 12:05:55.511741 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:55.511714    2580 scope.go:117] "RemoveContainer" containerID="6d2cca5ad1b4ac6d18428bc2f1c5deab6036324f035ee75aa9a1288f8f2210b9"
Sep 07 12:05:55.512110 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:05:55.512084    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d2cca5ad1b4ac6d18428bc2f1c5deab6036324f035ee75aa9a1288f8f2210b9\": container with ID starting with 6d2cca5ad1b4ac6d18428bc2f1c5deab6036324f035ee75aa9a1288f8f2210b9 not found: ID does not exist" containerID="6d2cca5ad1b4ac6d18428bc2f1c5deab6036324f035ee75aa9a1288f8f2210b9"
Sep 07 12:05:55.512191 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:55.512124    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d2cca5ad1b4ac6d18428bc2f1c5deab6036324f035ee75aa9a1288f8f2210b9"} err="failed to get container status \"6d2cca5ad1b4ac6d18428bc2f1c5deab6036324f035ee75aa9a1288f8f2210b9\": rpc error: code = NotFound desc = could not find container \"6d2cca5ad1b4ac6d18428bc2f1c5deab6036324f035ee75aa9a1288f8f2210b9\": container with ID starting with 6d2cca5ad1b4ac6d18428bc2f1c5deab6036324f035ee75aa9a1288f8f2210b9 not found: ID does not exist"
Sep 07 12:05:55.522949 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:55.522921    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"]
Sep 07 12:05:55.529279 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:55.529253    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c9822a85-8cf9f6486-6psrc"]
Sep 07 12:05:55.602630 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:55.602595    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"]
Sep 07 12:05:55.602935 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:55.602835    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk" podUID="a6741dff-46c8-47b6-b4f8-34bd8a3e349d" containerName="istio-proxy" containerID="cri-o://5309b0315f6fc735c08302593c664b8420000a7291f61e3263ad7e7232723227" gracePeriod=30
Sep 07 12:05:56.013445 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.011122    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ab214e9-cf9d-447b-999e-bb447f030cbf" path="/var/lib/kubelet/pods/2ab214e9-cf9d-447b-999e-bb447f030cbf/volumes"
Sep 07 12:05:56.013445 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.011757    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a630b7d-2458-48c4-b058-06873166fa26" path="/var/lib/kubelet/pods/6a630b7d-2458-48c4-b058-06873166fa26/volumes"
Sep 07 12:05:56.320527 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.320438    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"]
Sep 07 12:05:56.320901 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.320881    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ab214e9-cf9d-447b-999e-bb447f030cbf" containerName="revoke-keys"
Sep 07 12:05:56.320986 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.320904    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ab214e9-cf9d-447b-999e-bb447f030cbf" containerName="revoke-keys"
Sep 07 12:05:56.320986 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.320947    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a630b7d-2458-48c4-b058-06873166fa26" containerName="payload-pre-processing"
Sep 07 12:05:56.320986 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.320956    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a630b7d-2458-48c4-b058-06873166fa26" containerName="payload-pre-processing"
Sep 07 12:05:56.320986 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.320968    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58c3f85d-8417-4b47-afb8-a603c176018f" containerName="revoke-keys"
Sep 07 12:05:56.320986 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.320977    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="58c3f85d-8417-4b47-afb8-a603c176018f" containerName="revoke-keys"
Sep 07 12:05:56.321271 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.321069    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="58c3f85d-8417-4b47-afb8-a603c176018f" containerName="revoke-keys"
Sep 07 12:05:56.321271 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.321083    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a630b7d-2458-48c4-b058-06873166fa26" containerName="payload-pre-processing"
Sep 07 12:05:56.321271 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.321095    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ab214e9-cf9d-447b-999e-bb447f030cbf" containerName="revoke-keys"
Sep 07 12:05:56.324013 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.323991    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.326526 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.326501    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-57104ad7-openshift-default-dockercfg-zcdvl\""
Sep 07 12:05:56.335569 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.335539    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"]
Sep 07 12:05:56.342918 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.342888    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"]
Sep 07 12:05:56.343150 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.343125    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck" podUID="fc362ffb-cd70-42d1-b46a-4dcc7ee724a8" containerName="payload-pre-processing" containerID="cri-o://5b52bb1977cbba6d27fa995c565069d0cdf0e0fde54b1058f9c763d7172f0192" gracePeriod=30
Sep 07 12:05:56.457042 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.457001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-workload-socket\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.457247 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.457077    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istiod-ca-cert\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.457247 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.457117    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-credential-socket\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.457247 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.457148    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-token\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.457247 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.457172    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-envoy\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.457247 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.457221    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-workload-certs\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.457247 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.457241    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-podinfo\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.457622 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.457319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-data\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.457622 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.457346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sggx7\" (UniqueName: \"kubernetes.io/projected/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-kube-api-access-sggx7\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.507294 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.507262    2580 generic.go:356] "Generic (PLEG): container finished" podID="fc362ffb-cd70-42d1-b46a-4dcc7ee724a8" containerID="5b52bb1977cbba6d27fa995c565069d0cdf0e0fde54b1058f9c763d7172f0192" exitCode=0
Sep 07 12:05:56.507658 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.507332    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck" event={"ID":"fc362ffb-cd70-42d1-b46a-4dcc7ee724a8","Type":"ContainerDied","Data":"5b52bb1977cbba6d27fa995c565069d0cdf0e0fde54b1058f9c763d7172f0192"}
Sep 07 12:05:56.524806 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.524759    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk"]
Sep 07 12:05:56.527289 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.527261    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-ebf845-wphsk"]
Sep 07 12:05:56.558153 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.558123    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istiod-ca-cert\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.558343 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.558167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-credential-socket\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.558343 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.558217    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-token\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.558343 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.558245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-envoy\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.558343 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.558280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-workload-certs\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.558343 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.558307    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-podinfo\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.558603 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.558383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-data\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.558603 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.558413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sggx7\" (UniqueName: \"kubernetes.io/projected/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-kube-api-access-sggx7\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.558603 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.558490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-workload-socket\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.558760 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.558597    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-credential-socket\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.559139 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.559116    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-workload-socket\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.559291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.559158    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-workload-certs\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.559291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.559180    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istiod-ca-cert\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.559291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.559251    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-data\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.560789 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.560763    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-envoy\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.561012 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.560991    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-podinfo\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.566243 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.566211    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sggx7\" (UniqueName: \"kubernetes.io/projected/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-kube-api-access-sggx7\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.566674 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.566652    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-token\") pod \"e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") " pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.605122 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.605097    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"
Sep 07 12:05:56.642761 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.642728    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:56.662513 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.660257    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc362ffb-cd70-42d1-b46a-4dcc7ee724a8-plugins-config-volume\") pod \"fc362ffb-cd70-42d1-b46a-4dcc7ee724a8\" (UID: \"fc362ffb-cd70-42d1-b46a-4dcc7ee724a8\") "
Sep 07 12:05:56.662513 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.660338    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2m4h\" (UniqueName: \"kubernetes.io/projected/fc362ffb-cd70-42d1-b46a-4dcc7ee724a8-kube-api-access-p2m4h\") pod \"fc362ffb-cd70-42d1-b46a-4dcc7ee724a8\" (UID: \"fc362ffb-cd70-42d1-b46a-4dcc7ee724a8\") "
Sep 07 12:05:56.662513 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.661155    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc362ffb-cd70-42d1-b46a-4dcc7ee724a8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fc362ffb-cd70-42d1-b46a-4dcc7ee724a8" (UID: "fc362ffb-cd70-42d1-b46a-4dcc7ee724a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:05:56.664841 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.664743    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc362ffb-cd70-42d1-b46a-4dcc7ee724a8-kube-api-access-p2m4h" (OuterVolumeSpecName: "kube-api-access-p2m4h") pod "fc362ffb-cd70-42d1-b46a-4dcc7ee724a8" (UID: "fc362ffb-cd70-42d1-b46a-4dcc7ee724a8"). InnerVolumeSpecName "kube-api-access-p2m4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:05:56.761885 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.761841    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc362ffb-cd70-42d1-b46a-4dcc7ee724a8-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:05:56.761885 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.761876    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2m4h\" (UniqueName: \"kubernetes.io/projected/fc362ffb-cd70-42d1-b46a-4dcc7ee724a8-kube-api-access-p2m4h\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:05:56.940517 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.940480    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"]
Sep 07 12:05:56.943110 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:05:56.943078    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fcd8a3a_fd22_4432_9972_4ba06bf0f584.slice/crio-729fa8629f5a1f1c967b8198740afb552629598d9d1ed98b07b3532768a39cf4 WatchSource:0}: Error finding container 729fa8629f5a1f1c967b8198740afb552629598d9d1ed98b07b3532768a39cf4: Status 404 returned error can't find the container with id 729fa8629f5a1f1c967b8198740afb552629598d9d1ed98b07b3532768a39cf4
Sep 07 12:05:56.946016 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.945975    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:05:56.946112 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.946075    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:05:56.946151 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:56.946122    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:05:57.516492 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:57.516462    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"
Sep 07 12:05:57.516948 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:57.516507    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck" event={"ID":"fc362ffb-cd70-42d1-b46a-4dcc7ee724a8","Type":"ContainerDied","Data":"8ca344aa23053fad46d598c9ec47ea159e5822bbd53ca1175c6ca071c3ab77b1"}
Sep 07 12:05:57.516948 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:57.516575    2580 scope.go:117] "RemoveContainer" containerID="5b52bb1977cbba6d27fa995c565069d0cdf0e0fde54b1058f9c763d7172f0192"
Sep 07 12:05:57.518595 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:57.518567    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx" event={"ID":"7fcd8a3a-fd22-4432-9972-4ba06bf0f584","Type":"ContainerStarted","Data":"ce51b1f72dabae3819eaca0bbb27a7c0c550aed59d7358a861913e48b9c08821"}
Sep 07 12:05:57.518710 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:57.518611    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx" event={"ID":"7fcd8a3a-fd22-4432-9972-4ba06bf0f584","Type":"ContainerStarted","Data":"729fa8629f5a1f1c967b8198740afb552629598d9d1ed98b07b3532768a39cf4"}
Sep 07 12:05:57.541962 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:57.541907    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx" podStartSLOduration=1.5418874009999999 podStartE2EDuration="1.541887401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:05:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:05:57.539571288 +0000 UTC m=+1112.035460710" watchObservedRunningTime="2026-09-07 12:05:57.541887401 +0000 UTC m=+1112.037776824"
Sep 07 12:05:57.553964 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:57.553928    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"]
Sep 07 12:05:57.557486 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:57.557456    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-ebf845-f8ddd6546-rsbck"]
Sep 07 12:05:57.643051 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:57.642957    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:58.010187 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:58.010101    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58c3f85d-8417-4b47-afb8-a603c176018f" path="/var/lib/kubelet/pods/58c3f85d-8417-4b47-afb8-a603c176018f/volumes"
Sep 07 12:05:58.010518 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:58.010503    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc362ffb-cd70-42d1-b46a-4dcc7ee724a8" path="/var/lib/kubelet/pods/fc362ffb-cd70-42d1-b46a-4dcc7ee724a8/volumes"
Sep 07 12:05:58.649108 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:58.649081    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:59.530243 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:59.530190    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:05:59.531315 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:05:59.531291    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:06:00.860274 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.860238    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:06:00.890738 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.890700    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"]
Sep 07 12:06:00.891057 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.891045    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6741dff-46c8-47b6-b4f8-34bd8a3e349d" containerName="istio-proxy"
Sep 07 12:06:00.891107 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.891059    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6741dff-46c8-47b6-b4f8-34bd8a3e349d" containerName="istio-proxy"
Sep 07 12:06:00.891107 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.891076    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc362ffb-cd70-42d1-b46a-4dcc7ee724a8" containerName="payload-pre-processing"
Sep 07 12:06:00.891107 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.891081    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc362ffb-cd70-42d1-b46a-4dcc7ee724a8" containerName="payload-pre-processing"
Sep 07 12:06:00.891235 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.891139    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc362ffb-cd70-42d1-b46a-4dcc7ee724a8" containerName="payload-pre-processing"
Sep 07 12:06:00.891235 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.891151    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6741dff-46c8-47b6-b4f8-34bd8a3e349d" containerName="istio-proxy"
Sep 07 12:06:00.893099 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.893083    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"
Sep 07 12:06:00.895378 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.895350    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-57104ad7-dockercfg-qjk6b\""
Sep 07 12:06:00.895535 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.895516    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-57104ad7\""
Sep 07 12:06:00.904105 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.904072    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"]
Sep 07 12:06:00.909491 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.909465    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-podinfo\") pod \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") "
Sep 07 12:06:00.909594 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.909530    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-token\") pod \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") "
Sep 07 12:06:00.909594 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.909565    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-envoy\") pod \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") "
Sep 07 12:06:00.909723 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.909590    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-data\") pod \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") "
Sep 07 12:06:00.909723 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.909669    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-workload-socket\") pod \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") "
Sep 07 12:06:00.909723 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.909695    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdlpj\" (UniqueName: \"kubernetes.io/projected/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-kube-api-access-hdlpj\") pod \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") "
Sep 07 12:06:00.909857 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.909731    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-workload-certs\") pod \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") "
Sep 07 12:06:00.909857 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.909776    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-credential-socket\") pod \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") "
Sep 07 12:06:00.909857 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.909838    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istiod-ca-cert\") pod \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\" (UID: \"a6741dff-46c8-47b6-b4f8-34bd8a3e349d\") "
Sep 07 12:06:00.910294 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.910225    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a6741dff-46c8-47b6-b4f8-34bd8a3e349d" (UID: "a6741dff-46c8-47b6-b4f8-34bd8a3e349d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:00.910799 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.910501    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a6741dff-46c8-47b6-b4f8-34bd8a3e349d" (UID: "a6741dff-46c8-47b6-b4f8-34bd8a3e349d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:00.910799 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.910531    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a6741dff-46c8-47b6-b4f8-34bd8a3e349d" (UID: "a6741dff-46c8-47b6-b4f8-34bd8a3e349d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:06:00.910799 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.910703    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a6741dff-46c8-47b6-b4f8-34bd8a3e349d" (UID: "a6741dff-46c8-47b6-b4f8-34bd8a3e349d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:00.911028 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.910978    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-data" (OuterVolumeSpecName: "istio-data") pod "a6741dff-46c8-47b6-b4f8-34bd8a3e349d" (UID: "a6741dff-46c8-47b6-b4f8-34bd8a3e349d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:00.934369 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.929345    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a6741dff-46c8-47b6-b4f8-34bd8a3e349d" (UID: "a6741dff-46c8-47b6-b4f8-34bd8a3e349d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:06:00.934369 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.930712    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a6741dff-46c8-47b6-b4f8-34bd8a3e349d" (UID: "a6741dff-46c8-47b6-b4f8-34bd8a3e349d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:00.934369 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.931242    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-kube-api-access-hdlpj" (OuterVolumeSpecName: "kube-api-access-hdlpj") pod "a6741dff-46c8-47b6-b4f8-34bd8a3e349d" (UID: "a6741dff-46c8-47b6-b4f8-34bd8a3e349d"). InnerVolumeSpecName "kube-api-access-hdlpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:06:00.934983 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:00.934938    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-token" (OuterVolumeSpecName: "istio-token") pod "a6741dff-46c8-47b6-b4f8-34bd8a3e349d" (UID: "a6741dff-46c8-47b6-b4f8-34bd8a3e349d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:06:01.011104 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.011066    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bac6388-971f-449b-893f-d1d1d6dca84d-plugins-config-volume\") pod \"payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj\" (UID: \"1bac6388-971f-449b-893f-d1d1d6dca84d\") " pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"
Sep 07 12:06:01.011308 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.011116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bndp7\" (UniqueName: \"kubernetes.io/projected/1bac6388-971f-449b-893f-d1d1d6dca84d-kube-api-access-bndp7\") pod \"payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj\" (UID: \"1bac6388-971f-449b-893f-d1d1d6dca84d\") " pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"
Sep 07 12:06:01.011308 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.011175    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-token\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:01.011308 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.011187    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-envoy\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:01.011308 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.011194    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-data\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:01.011308 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.011222    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-workload-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:01.011308 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.011235    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdlpj\" (UniqueName: \"kubernetes.io/projected/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-kube-api-access-hdlpj\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:01.011308 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.011243    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-workload-certs\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:01.011308 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.011253    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-credential-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:01.011308 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.011261    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istiod-ca-cert\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:01.011308 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.011268    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6741dff-46c8-47b6-b4f8-34bd8a3e349d-istio-podinfo\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:01.111797 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.111743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bac6388-971f-449b-893f-d1d1d6dca84d-plugins-config-volume\") pod \"payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj\" (UID: \"1bac6388-971f-449b-893f-d1d1d6dca84d\") " pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"
Sep 07 12:06:01.111982 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.111848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bndp7\" (UniqueName: \"kubernetes.io/projected/1bac6388-971f-449b-893f-d1d1d6dca84d-kube-api-access-bndp7\") pod \"payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj\" (UID: \"1bac6388-971f-449b-893f-d1d1d6dca84d\") " pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"
Sep 07 12:06:01.112494 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.112474    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bac6388-971f-449b-893f-d1d1d6dca84d-plugins-config-volume\") pod \"payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj\" (UID: \"1bac6388-971f-449b-893f-d1d1d6dca84d\") " pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"
Sep 07 12:06:01.119631 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.119610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bndp7\" (UniqueName: \"kubernetes.io/projected/1bac6388-971f-449b-893f-d1d1d6dca84d-kube-api-access-bndp7\") pod \"payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj\" (UID: \"1bac6388-971f-449b-893f-d1d1d6dca84d\") " pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"
Sep 07 12:06:01.207021 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.206935    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"
Sep 07 12:06:01.476871 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.476843    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"]
Sep 07 12:06:01.479480 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:06:01.479452    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bac6388_971f_449b_893f_d1d1d6dca84d.slice/crio-446667751a57c579b313d7b8034e2b05090623d788b478650896a5b69970c485 WatchSource:0}: Error finding container 446667751a57c579b313d7b8034e2b05090623d788b478650896a5b69970c485: Status 404 returned error can't find the container with id 446667751a57c579b313d7b8034e2b05090623d788b478650896a5b69970c485
Sep 07 12:06:01.541907 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.541875    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj" event={"ID":"1bac6388-971f-449b-893f-d1d1d6dca84d","Type":"ContainerStarted","Data":"446667751a57c579b313d7b8034e2b05090623d788b478650896a5b69970c485"}
Sep 07 12:06:01.543665 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.543638    2580 generic.go:356] "Generic (PLEG): container finished" podID="a6741dff-46c8-47b6-b4f8-34bd8a3e349d" containerID="5309b0315f6fc735c08302593c664b8420000a7291f61e3263ad7e7232723227" exitCode=0
Sep 07 12:06:01.543782 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.543677    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"
Sep 07 12:06:01.543782 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.543720    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk" event={"ID":"a6741dff-46c8-47b6-b4f8-34bd8a3e349d","Type":"ContainerDied","Data":"5309b0315f6fc735c08302593c664b8420000a7291f61e3263ad7e7232723227"}
Sep 07 12:06:01.543782 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.543777    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk" event={"ID":"a6741dff-46c8-47b6-b4f8-34bd8a3e349d","Type":"ContainerDied","Data":"22a8b739e13979b9bcc5bb672ccb8e25575de253751eea9d2bb76eaaebd947a7"}
Sep 07 12:06:01.543945 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.543808    2580 scope.go:117] "RemoveContainer" containerID="5309b0315f6fc735c08302593c664b8420000a7291f61e3263ad7e7232723227"
Sep 07 12:06:01.552640 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.552612    2580 scope.go:117] "RemoveContainer" containerID="4703a330df8afc6fa40e4b3375cda893bfc4cb0c2c8fa20575ac3f541b230682"
Sep 07 12:06:01.561327 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.561304    2580 scope.go:117] "RemoveContainer" containerID="5309b0315f6fc735c08302593c664b8420000a7291f61e3263ad7e7232723227"
Sep 07 12:06:01.561614 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:06:01.561594    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5309b0315f6fc735c08302593c664b8420000a7291f61e3263ad7e7232723227\": container with ID starting with 5309b0315f6fc735c08302593c664b8420000a7291f61e3263ad7e7232723227 not found: ID does not exist" containerID="5309b0315f6fc735c08302593c664b8420000a7291f61e3263ad7e7232723227"
Sep 07 12:06:01.561693 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.561621    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5309b0315f6fc735c08302593c664b8420000a7291f61e3263ad7e7232723227"} err="failed to get container status \"5309b0315f6fc735c08302593c664b8420000a7291f61e3263ad7e7232723227\": rpc error: code = NotFound desc = could not find container \"5309b0315f6fc735c08302593c664b8420000a7291f61e3263ad7e7232723227\": container with ID starting with 5309b0315f6fc735c08302593c664b8420000a7291f61e3263ad7e7232723227 not found: ID does not exist"
Sep 07 12:06:01.561693 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.561647    2580 scope.go:117] "RemoveContainer" containerID="4703a330df8afc6fa40e4b3375cda893bfc4cb0c2c8fa20575ac3f541b230682"
Sep 07 12:06:01.561934 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:06:01.561915    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4703a330df8afc6fa40e4b3375cda893bfc4cb0c2c8fa20575ac3f541b230682\": container with ID starting with 4703a330df8afc6fa40e4b3375cda893bfc4cb0c2c8fa20575ac3f541b230682 not found: ID does not exist" containerID="4703a330df8afc6fa40e4b3375cda893bfc4cb0c2c8fa20575ac3f541b230682"
Sep 07 12:06:01.562005 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.561939    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4703a330df8afc6fa40e4b3375cda893bfc4cb0c2c8fa20575ac3f541b230682"} err="failed to get container status \"4703a330df8afc6fa40e4b3375cda893bfc4cb0c2c8fa20575ac3f541b230682\": rpc error: code = NotFound desc = could not find container \"4703a330df8afc6fa40e4b3375cda893bfc4cb0c2c8fa20575ac3f541b230682\": container with ID starting with 4703a330df8afc6fa40e4b3375cda893bfc4cb0c2c8fa20575ac3f541b230682 not found: ID does not exist"
Sep 07 12:06:01.567859 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.567828    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"]
Sep 07 12:06:01.569530 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:01.569503    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c9822a85-openshift-default-7b6cf9595b-fgtkk"]
Sep 07 12:06:02.010468 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:02.010430    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6741dff-46c8-47b6-b4f8-34bd8a3e349d" path="/var/lib/kubelet/pods/a6741dff-46c8-47b6-b4f8-34bd8a3e349d/volumes"
Sep 07 12:06:02.551876 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:02.551844    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj" event={"ID":"1bac6388-971f-449b-893f-d1d1d6dca84d","Type":"ContainerStarted","Data":"4a8c70607d235a16c659ee172d6d0ec7b3363af983d1330f7db981783a22351e"}
Sep 07 12:06:02.552060 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:02.551962    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"
Sep 07 12:06:02.569835 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:02.569769    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj" podStartSLOduration=2.569738747 podStartE2EDuration="2.569738747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:06:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:06:02.569263054 +0000 UTC m=+1117.065152478" watchObservedRunningTime="2026-09-07 12:06:02.569738747 +0000 UTC m=+1117.065628171"
Sep 07 12:06:03.086576 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:03.086542    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"]
Sep 07 12:06:03.087025 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:03.086798    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg" podUID="9bb2c73f-959f-488f-83e9-19a8a7537103" containerName="istio-proxy" containerID="cri-o://98bed0f28db017bfa4ba90cf006e20bc286cb7e509878881872b4249b1ee373f" gracePeriod=30
Sep 07 12:06:08.378930 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.378898    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:06:08.488690 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.488651    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-credential-socket\") pod \"9bb2c73f-959f-488f-83e9-19a8a7537103\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") "
Sep 07 12:06:08.488840 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.488719    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-workload-certs\") pod \"9bb2c73f-959f-488f-83e9-19a8a7537103\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") "
Sep 07 12:06:08.488840 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.488746    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-podinfo\") pod \"9bb2c73f-959f-488f-83e9-19a8a7537103\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") "
Sep 07 12:06:08.488840 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.488770    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-data\") pod \"9bb2c73f-959f-488f-83e9-19a8a7537103\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") "
Sep 07 12:06:08.488958 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.488844    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-envoy\") pod \"9bb2c73f-959f-488f-83e9-19a8a7537103\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") "
Sep 07 12:06:08.488958 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.488874    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-workload-socket\") pod \"9bb2c73f-959f-488f-83e9-19a8a7537103\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") "
Sep 07 12:06:08.488958 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.488919    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndprw\" (UniqueName: \"kubernetes.io/projected/9bb2c73f-959f-488f-83e9-19a8a7537103-kube-api-access-ndprw\") pod \"9bb2c73f-959f-488f-83e9-19a8a7537103\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") "
Sep 07 12:06:08.488958 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.488947    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bb2c73f-959f-488f-83e9-19a8a7537103-istiod-ca-cert\") pod \"9bb2c73f-959f-488f-83e9-19a8a7537103\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") "
Sep 07 12:06:08.489076 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.488955    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9bb2c73f-959f-488f-83e9-19a8a7537103" (UID: "9bb2c73f-959f-488f-83e9-19a8a7537103"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:08.489076 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.488982    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9bb2c73f-959f-488f-83e9-19a8a7537103" (UID: "9bb2c73f-959f-488f-83e9-19a8a7537103"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:08.489142 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.489092    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-data" (OuterVolumeSpecName: "istio-data") pod "9bb2c73f-959f-488f-83e9-19a8a7537103" (UID: "9bb2c73f-959f-488f-83e9-19a8a7537103"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:08.489177 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.489135    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9bb2c73f-959f-488f-83e9-19a8a7537103" (UID: "9bb2c73f-959f-488f-83e9-19a8a7537103"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:08.489297 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.489272    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-token\") pod \"9bb2c73f-959f-488f-83e9-19a8a7537103\" (UID: \"9bb2c73f-959f-488f-83e9-19a8a7537103\") "
Sep 07 12:06:08.489435 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.489329    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bb2c73f-959f-488f-83e9-19a8a7537103-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9bb2c73f-959f-488f-83e9-19a8a7537103" (UID: "9bb2c73f-959f-488f-83e9-19a8a7537103"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:06:08.489662 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.489640    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-workload-certs\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:08.489787 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.489667    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-data\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:08.489787 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.489682    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-workload-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:08.489787 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.489704    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bb2c73f-959f-488f-83e9-19a8a7537103-istiod-ca-cert\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:08.489787 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.489719    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-credential-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:08.491449 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.491425    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-token" (OuterVolumeSpecName: "istio-token") pod "9bb2c73f-959f-488f-83e9-19a8a7537103" (UID: "9bb2c73f-959f-488f-83e9-19a8a7537103"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:06:08.491549 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.491441    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9bb2c73f-959f-488f-83e9-19a8a7537103" (UID: "9bb2c73f-959f-488f-83e9-19a8a7537103"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:08.491743 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.491722    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bb2c73f-959f-488f-83e9-19a8a7537103-kube-api-access-ndprw" (OuterVolumeSpecName: "kube-api-access-ndprw") pod "9bb2c73f-959f-488f-83e9-19a8a7537103" (UID: "9bb2c73f-959f-488f-83e9-19a8a7537103"). InnerVolumeSpecName "kube-api-access-ndprw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:06:08.491865 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.491736    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9bb2c73f-959f-488f-83e9-19a8a7537103" (UID: "9bb2c73f-959f-488f-83e9-19a8a7537103"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:06:08.544247 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.544182    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"]
Sep 07 12:06:08.544635 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.544618    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bb2c73f-959f-488f-83e9-19a8a7537103" containerName="istio-proxy"
Sep 07 12:06:08.544734 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.544639    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bb2c73f-959f-488f-83e9-19a8a7537103" containerName="istio-proxy"
Sep 07 12:06:08.544734 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.544656    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6741dff-46c8-47b6-b4f8-34bd8a3e349d" containerName="istio-proxy"
Sep 07 12:06:08.544734 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.544664    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6741dff-46c8-47b6-b4f8-34bd8a3e349d" containerName="istio-proxy"
Sep 07 12:06:08.544734 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.544686    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bb2c73f-959f-488f-83e9-19a8a7537103" containerName="istio-proxy"
Sep 07 12:06:08.544734 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.544695    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bb2c73f-959f-488f-83e9-19a8a7537103" containerName="istio-proxy"
Sep 07 12:06:08.544980 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.544775    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bb2c73f-959f-488f-83e9-19a8a7537103" containerName="istio-proxy"
Sep 07 12:06:08.544980 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.544786    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6741dff-46c8-47b6-b4f8-34bd8a3e349d" containerName="istio-proxy"
Sep 07 12:06:08.547165 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.547137    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"
Sep 07 12:06:08.564735 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.564690    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"]
Sep 07 12:06:08.564975 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.564954    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-3fa352-dockercfg-scpf4\""
Sep 07 12:06:08.565945 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.565918    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-3fa352\""
Sep 07 12:06:08.590271 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.590243    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-envoy\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:08.590484 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.590466    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndprw\" (UniqueName: \"kubernetes.io/projected/9bb2c73f-959f-488f-83e9-19a8a7537103-kube-api-access-ndprw\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:08.590603 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.590589    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-token\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:08.590691 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.590680    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bb2c73f-959f-488f-83e9-19a8a7537103-istio-podinfo\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:08.591427 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.591402    2580 generic.go:356] "Generic (PLEG): container finished" podID="9bb2c73f-959f-488f-83e9-19a8a7537103" containerID="98bed0f28db017bfa4ba90cf006e20bc286cb7e509878881872b4249b1ee373f" exitCode=0
Sep 07 12:06:08.591624 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.591492    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"
Sep 07 12:06:08.591719 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.591504    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg" event={"ID":"9bb2c73f-959f-488f-83e9-19a8a7537103","Type":"ContainerDied","Data":"98bed0f28db017bfa4ba90cf006e20bc286cb7e509878881872b4249b1ee373f"}
Sep 07 12:06:08.591760 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.591738    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg" event={"ID":"9bb2c73f-959f-488f-83e9-19a8a7537103","Type":"ContainerDied","Data":"5189d5daafd1811d1336169de6aef64a576bf6bdcb6a0663f53137037c067a23"}
Sep 07 12:06:08.591797 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.591771    2580 scope.go:117] "RemoveContainer" containerID="98bed0f28db017bfa4ba90cf006e20bc286cb7e509878881872b4249b1ee373f"
Sep 07 12:06:08.601397 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.601379    2580 scope.go:117] "RemoveContainer" containerID="95097cf281ec4cc267d78d94597dbc679eccd6839e5d6374ed04c16d6df70883"
Sep 07 12:06:08.609591 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.609567    2580 scope.go:117] "RemoveContainer" containerID="98bed0f28db017bfa4ba90cf006e20bc286cb7e509878881872b4249b1ee373f"
Sep 07 12:06:08.609942 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:06:08.609923    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98bed0f28db017bfa4ba90cf006e20bc286cb7e509878881872b4249b1ee373f\": container with ID starting with 98bed0f28db017bfa4ba90cf006e20bc286cb7e509878881872b4249b1ee373f not found: ID does not exist" containerID="98bed0f28db017bfa4ba90cf006e20bc286cb7e509878881872b4249b1ee373f"
Sep 07 12:06:08.610030 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.609955    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98bed0f28db017bfa4ba90cf006e20bc286cb7e509878881872b4249b1ee373f"} err="failed to get container status \"98bed0f28db017bfa4ba90cf006e20bc286cb7e509878881872b4249b1ee373f\": rpc error: code = NotFound desc = could not find container \"98bed0f28db017bfa4ba90cf006e20bc286cb7e509878881872b4249b1ee373f\": container with ID starting with 98bed0f28db017bfa4ba90cf006e20bc286cb7e509878881872b4249b1ee373f not found: ID does not exist"
Sep 07 12:06:08.610030 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.609982    2580 scope.go:117] "RemoveContainer" containerID="95097cf281ec4cc267d78d94597dbc679eccd6839e5d6374ed04c16d6df70883"
Sep 07 12:06:08.610313 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:06:08.610290    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95097cf281ec4cc267d78d94597dbc679eccd6839e5d6374ed04c16d6df70883\": container with ID starting with 95097cf281ec4cc267d78d94597dbc679eccd6839e5d6374ed04c16d6df70883 not found: ID does not exist" containerID="95097cf281ec4cc267d78d94597dbc679eccd6839e5d6374ed04c16d6df70883"
Sep 07 12:06:08.610353 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.610323    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95097cf281ec4cc267d78d94597dbc679eccd6839e5d6374ed04c16d6df70883"} err="failed to get container status \"95097cf281ec4cc267d78d94597dbc679eccd6839e5d6374ed04c16d6df70883\": rpc error: code = NotFound desc = could not find container \"95097cf281ec4cc267d78d94597dbc679eccd6839e5d6374ed04c16d6df70883\": container with ID starting with 95097cf281ec4cc267d78d94597dbc679eccd6839e5d6374ed04c16d6df70883 not found: ID does not exist"
Sep 07 12:06:08.613295 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.613272    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"]
Sep 07 12:06:08.616552 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.616529    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-ebf845-openshift-default-849fc6b478-xhhsg"]
Sep 07 12:06:08.691902 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.691748    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2edce9b-0dde-49b9-8c36-5e1f217bd2ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85\" (UID: \"e2edce9b-0dde-49b9-8c36-5e1f217bd2ad\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"
Sep 07 12:06:08.691902 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.691849    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97cfx\" (UniqueName: \"kubernetes.io/projected/e2edce9b-0dde-49b9-8c36-5e1f217bd2ad-kube-api-access-97cfx\") pod \"payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85\" (UID: \"e2edce9b-0dde-49b9-8c36-5e1f217bd2ad\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"
Sep 07 12:06:08.793089 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.793055    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2edce9b-0dde-49b9-8c36-5e1f217bd2ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85\" (UID: \"e2edce9b-0dde-49b9-8c36-5e1f217bd2ad\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"
Sep 07 12:06:08.793089 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.793092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97cfx\" (UniqueName: \"kubernetes.io/projected/e2edce9b-0dde-49b9-8c36-5e1f217bd2ad-kube-api-access-97cfx\") pod \"payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85\" (UID: \"e2edce9b-0dde-49b9-8c36-5e1f217bd2ad\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"
Sep 07 12:06:08.793891 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.793864    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2edce9b-0dde-49b9-8c36-5e1f217bd2ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85\" (UID: \"e2edce9b-0dde-49b9-8c36-5e1f217bd2ad\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"
Sep 07 12:06:08.800728 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.800701    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97cfx\" (UniqueName: \"kubernetes.io/projected/e2edce9b-0dde-49b9-8c36-5e1f217bd2ad-kube-api-access-97cfx\") pod \"payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85\" (UID: \"e2edce9b-0dde-49b9-8c36-5e1f217bd2ad\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"
Sep 07 12:06:08.885395 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:08.885353    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"
Sep 07 12:06:09.162221 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:09.161299    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"]
Sep 07 12:06:09.598494 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:09.598386    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85" event={"ID":"e2edce9b-0dde-49b9-8c36-5e1f217bd2ad","Type":"ContainerStarted","Data":"5e3f4b21b93b89bfe3ee2284c63e248d85d09cad12a140ea1e1f77d3055f7131"}
Sep 07 12:06:09.598494 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:09.598460    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"
Sep 07 12:06:09.598494 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:09.598485    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85" event={"ID":"e2edce9b-0dde-49b9-8c36-5e1f217bd2ad","Type":"ContainerStarted","Data":"3e4cf7c700f32558d4146b09cf26f47fcb943aa2a293186a5a679d9d3c348ed0"}
Sep 07 12:06:09.614751 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:09.614684    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85" podStartSLOduration=1.614665521 podStartE2EDuration="1.614665521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:06:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:06:09.613091894 +0000 UTC m=+1124.108981318" watchObservedRunningTime="2026-09-07 12:06:09.614665521 +0000 UTC m=+1124.110554943"
Sep 07 12:06:10.010405 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:10.010371    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bb2c73f-959f-488f-83e9-19a8a7537103" path="/var/lib/kubelet/pods/9bb2c73f-959f-488f-83e9-19a8a7537103/volumes"
Sep 07 12:06:13.559386 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:13.559356    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"
Sep 07 12:06:14.531855 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:14.531764    2580 patch_prober.go:28] interesting pod/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 07 12:06:14.531855 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:14.531823    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx" podUID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 07 12:06:14.628491 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:14.628449    2580 generic.go:356] "Generic (PLEG): container finished" podID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" containerID="ce51b1f72dabae3819eaca0bbb27a7c0c550aed59d7358a861913e48b9c08821" exitCode=0
Sep 07 12:06:14.628982 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:14.628526    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx" event={"ID":"7fcd8a3a-fd22-4432-9972-4ba06bf0f584","Type":"ContainerDied","Data":"ce51b1f72dabae3819eaca0bbb27a7c0c550aed59d7358a861913e48b9c08821"}
Sep 07 12:06:14.629129 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:14.629110    2580 scope.go:117] "RemoveContainer" containerID="ce51b1f72dabae3819eaca0bbb27a7c0c550aed59d7358a861913e48b9c08821"
Sep 07 12:06:14.630398 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:14.630366    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:06:14.630475 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:14.630451    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:06:14.630511 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:14.630484    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:06:15.635644 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:15.635612    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx" event={"ID":"7fcd8a3a-fd22-4432-9972-4ba06bf0f584","Type":"ContainerStarted","Data":"97df18bfc7963e1ad4d4912844fd2795cac3d5233a995a72c64ae6ec80733bc3"}
Sep 07 12:06:15.643373 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:15.643335    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:06:15.643610 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:15.643585    2580 patch_prober.go:28] interesting pod/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 07 12:06:15.643687 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:15.643636    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx" podUID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 07 12:06:16.643557 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:16.643519    2580 patch_prober.go:28] interesting pod/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 07 12:06:16.643942 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:16.643571    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx" podUID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 07 12:06:17.643214 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:17.643171    2580 patch_prober.go:28] interesting pod/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 07 12:06:17.643408 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:17.643242    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx" podUID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 07 12:06:18.643821 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:18.643783    2580 patch_prober.go:28] interesting pod/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 07 12:06:18.644258 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:18.643838    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx" podUID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 07 12:06:19.643727 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:19.643688    2580 patch_prober.go:28] interesting pod/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 07 12:06:19.643910 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:19.643745    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx" podUID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 07 12:06:20.607357 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:20.607314    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"
Sep 07 12:06:20.804887 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:20.804856    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:06:20.805457 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:20.805237    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:06:20.806185 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:20.806165    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:06:23.688337 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:23.688298    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw"]
Sep 07 12:06:23.689140 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:23.688948    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bb2c73f-959f-488f-83e9-19a8a7537103" containerName="istio-proxy"
Sep 07 12:06:23.690927 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:23.690904    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw"
Sep 07 12:06:23.693403 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:23.693380    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:06:23.693403 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:23.693382    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:06:23.698292 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:23.698251    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw"]
Sep 07 12:06:23.848274 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:23.848232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ef098fb-e205-444a-b97b-cb9a9746130e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw\" (UID: \"9ef098fb-e205-444a-b97b-cb9a9746130e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw"
Sep 07 12:06:23.949399 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:23.949295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ef098fb-e205-444a-b97b-cb9a9746130e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw\" (UID: \"9ef098fb-e205-444a-b97b-cb9a9746130e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw"
Sep 07 12:06:23.949936 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:23.949916    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ef098fb-e205-444a-b97b-cb9a9746130e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw\" (UID: \"9ef098fb-e205-444a-b97b-cb9a9746130e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw"
Sep 07 12:06:24.003300 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:24.003257    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw"
Sep 07 12:06:24.272937 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:24.272910    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw"]
Sep 07 12:06:24.274932 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:06:24.274901    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ef098fb_e205_444a_b97b_cb9a9746130e.slice/crio-2f178e3d640b6bb921b67a8ce485b98118422580f3dd7a42269be742fcae3227 WatchSource:0}: Error finding container 2f178e3d640b6bb921b67a8ce485b98118422580f3dd7a42269be742fcae3227: Status 404 returned error can't find the container with id 2f178e3d640b6bb921b67a8ce485b98118422580f3dd7a42269be742fcae3227
Sep 07 12:06:24.689088 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:24.689052    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw" event={"ID":"9ef098fb-e205-444a-b97b-cb9a9746130e","Type":"ContainerStarted","Data":"23815a1fcf8938a88ff9b54b75c290da4c17b164d52fd77d9f80bbbdef5f621a"}
Sep 07 12:06:24.689558 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:24.689103    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw" event={"ID":"9ef098fb-e205-444a-b97b-cb9a9746130e","Type":"ContainerStarted","Data":"2f178e3d640b6bb921b67a8ce485b98118422580f3dd7a42269be742fcae3227"}
Sep 07 12:06:24.705808 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:24.705732    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw" podStartSLOduration=1.7057099 podStartE2EDuration="1.7057099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:06:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:06:24.70501674 +0000 UTC m=+1139.200906164" watchObservedRunningTime="2026-09-07 12:06:24.7057099 +0000 UTC m=+1139.201599325"
Sep 07 12:06:25.696334 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:25.696297    2580 generic.go:356] "Generic (PLEG): container finished" podID="9ef098fb-e205-444a-b97b-cb9a9746130e" containerID="23815a1fcf8938a88ff9b54b75c290da4c17b164d52fd77d9f80bbbdef5f621a" exitCode=0
Sep 07 12:06:25.696725 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:25.696385    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw" event={"ID":"9ef098fb-e205-444a-b97b-cb9a9746130e","Type":"ContainerDied","Data":"23815a1fcf8938a88ff9b54b75c290da4c17b164d52fd77d9f80bbbdef5f621a"}
Sep 07 12:06:26.835018 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:26.834993    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw"
Sep 07 12:06:26.979874 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:26.979773    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ef098fb-e205-444a-b97b-cb9a9746130e-maas-api-service-ca\") pod \"9ef098fb-e205-444a-b97b-cb9a9746130e\" (UID: \"9ef098fb-e205-444a-b97b-cb9a9746130e\") "
Sep 07 12:06:26.980145 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:26.980118    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ef098fb-e205-444a-b97b-cb9a9746130e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9ef098fb-e205-444a-b97b-cb9a9746130e" (UID: "9ef098fb-e205-444a-b97b-cb9a9746130e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:06:27.081437 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:27.081392    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ef098fb-e205-444a-b97b-cb9a9746130e-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:27.707226 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:27.707178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw" event={"ID":"9ef098fb-e205-444a-b97b-cb9a9746130e","Type":"ContainerDied","Data":"2f178e3d640b6bb921b67a8ce485b98118422580f3dd7a42269be742fcae3227"}
Sep 07 12:06:27.707226 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:27.707194    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw"
Sep 07 12:06:27.707490 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:27.707240    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f178e3d640b6bb921b67a8ce485b98118422580f3dd7a42269be742fcae3227"
Sep 07 12:06:30.428945 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.428904    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"]
Sep 07 12:06:30.429439 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.429143    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj" podUID="1bac6388-971f-449b-893f-d1d1d6dca84d" containerName="payload-processing" containerID="cri-o://4a8c70607d235a16c659ee172d6d0ec7b3363af983d1330f7db981783a22351e" gracePeriod=30
Sep 07 12:06:30.588922 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.588886    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw"]
Sep 07 12:06:30.590659 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.590634    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57104ad7-8ptjw"]
Sep 07 12:06:30.694121 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.693976    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"
Sep 07 12:06:30.726016 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.725983    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"]
Sep 07 12:06:30.726437 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.726296    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx" podUID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" containerName="istio-proxy" containerID="cri-o://97df18bfc7963e1ad4d4912844fd2795cac3d5233a995a72c64ae6ec80733bc3" gracePeriod=30
Sep 07 12:06:30.729048 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.729016    2580 generic.go:356] "Generic (PLEG): container finished" podID="1bac6388-971f-449b-893f-d1d1d6dca84d" containerID="4a8c70607d235a16c659ee172d6d0ec7b3363af983d1330f7db981783a22351e" exitCode=0
Sep 07 12:06:30.729171 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.729092    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj" event={"ID":"1bac6388-971f-449b-893f-d1d1d6dca84d","Type":"ContainerDied","Data":"4a8c70607d235a16c659ee172d6d0ec7b3363af983d1330f7db981783a22351e"}
Sep 07 12:06:30.729171 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.729142    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj" event={"ID":"1bac6388-971f-449b-893f-d1d1d6dca84d","Type":"ContainerDied","Data":"446667751a57c579b313d7b8034e2b05090623d788b478650896a5b69970c485"}
Sep 07 12:06:30.729331 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.729175    2580 scope.go:117] "RemoveContainer" containerID="4a8c70607d235a16c659ee172d6d0ec7b3363af983d1330f7db981783a22351e"
Sep 07 12:06:30.729830 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.729359    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"
Sep 07 12:06:30.739086 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.739060    2580 scope.go:117] "RemoveContainer" containerID="4a8c70607d235a16c659ee172d6d0ec7b3363af983d1330f7db981783a22351e"
Sep 07 12:06:30.739667 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:06:30.739641    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a8c70607d235a16c659ee172d6d0ec7b3363af983d1330f7db981783a22351e\": container with ID starting with 4a8c70607d235a16c659ee172d6d0ec7b3363af983d1330f7db981783a22351e not found: ID does not exist" containerID="4a8c70607d235a16c659ee172d6d0ec7b3363af983d1330f7db981783a22351e"
Sep 07 12:06:30.739786 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.739675    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a8c70607d235a16c659ee172d6d0ec7b3363af983d1330f7db981783a22351e"} err="failed to get container status \"4a8c70607d235a16c659ee172d6d0ec7b3363af983d1330f7db981783a22351e\": rpc error: code = NotFound desc = could not find container \"4a8c70607d235a16c659ee172d6d0ec7b3363af983d1330f7db981783a22351e\": container with ID starting with 4a8c70607d235a16c659ee172d6d0ec7b3363af983d1330f7db981783a22351e not found: ID does not exist"
Sep 07 12:06:30.827510 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.827470    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bac6388-971f-449b-893f-d1d1d6dca84d-plugins-config-volume\") pod \"1bac6388-971f-449b-893f-d1d1d6dca84d\" (UID: \"1bac6388-971f-449b-893f-d1d1d6dca84d\") "
Sep 07 12:06:30.827510 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.827522    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bndp7\" (UniqueName: \"kubernetes.io/projected/1bac6388-971f-449b-893f-d1d1d6dca84d-kube-api-access-bndp7\") pod \"1bac6388-971f-449b-893f-d1d1d6dca84d\" (UID: \"1bac6388-971f-449b-893f-d1d1d6dca84d\") "
Sep 07 12:06:30.827906 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.827879    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bac6388-971f-449b-893f-d1d1d6dca84d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1bac6388-971f-449b-893f-d1d1d6dca84d" (UID: "1bac6388-971f-449b-893f-d1d1d6dca84d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:06:30.829597 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.829566    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bac6388-971f-449b-893f-d1d1d6dca84d-kube-api-access-bndp7" (OuterVolumeSpecName: "kube-api-access-bndp7") pod "1bac6388-971f-449b-893f-d1d1d6dca84d" (UID: "1bac6388-971f-449b-893f-d1d1d6dca84d"). InnerVolumeSpecName "kube-api-access-bndp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:06:30.928632 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.928599    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bac6388-971f-449b-893f-d1d1d6dca84d-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:30.928632 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:30.928629    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bndp7\" (UniqueName: \"kubernetes.io/projected/1bac6388-971f-449b-893f-d1d1d6dca84d-kube-api-access-bndp7\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:31.051625 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:31.051564    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"]
Sep 07 12:06:31.053144 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:31.053120    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-57104ad7-644c8f788f-5rpkj"]
Sep 07 12:06:32.010131 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:32.010095    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bac6388-971f-449b-893f-d1d1d6dca84d" path="/var/lib/kubelet/pods/1bac6388-971f-449b-893f-d1d1d6dca84d/volumes"
Sep 07 12:06:32.010547 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:32.010486    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ef098fb-e205-444a-b97b-cb9a9746130e" path="/var/lib/kubelet/pods/9ef098fb-e205-444a-b97b-cb9a9746130e/volumes"
Sep 07 12:06:35.806930 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:35.806895    2580 patch_prober.go:28] interesting pod/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 07 12:06:35.807310 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:35.806958    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx" podUID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 07 12:06:35.975439 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:35.975412    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:06:36.071025 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.070931    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sggx7\" (UniqueName: \"kubernetes.io/projected/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-kube-api-access-sggx7\") pod \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") "
Sep 07 12:06:36.071025 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.070976    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-podinfo\") pod \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") "
Sep 07 12:06:36.071025 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.071020    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-data\") pod \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") "
Sep 07 12:06:36.071334 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.071123    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-workload-socket\") pod \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") "
Sep 07 12:06:36.071334 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.071166    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-token\") pod \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") "
Sep 07 12:06:36.071334 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.071224    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-envoy\") pod \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") "
Sep 07 12:06:36.071334 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.071296    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-credential-socket\") pod \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") "
Sep 07 12:06:36.071334 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.071322    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-data" (OuterVolumeSpecName: "istio-data") pod "7fcd8a3a-fd22-4432-9972-4ba06bf0f584" (UID: "7fcd8a3a-fd22-4432-9972-4ba06bf0f584"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:36.071576 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.071342    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-workload-certs\") pod \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") "
Sep 07 12:06:36.071576 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.071387    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istiod-ca-cert\") pod \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\" (UID: \"7fcd8a3a-fd22-4432-9972-4ba06bf0f584\") "
Sep 07 12:06:36.071576 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.071410    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7fcd8a3a-fd22-4432-9972-4ba06bf0f584" (UID: "7fcd8a3a-fd22-4432-9972-4ba06bf0f584"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:36.071793 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.071586    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7fcd8a3a-fd22-4432-9972-4ba06bf0f584" (UID: "7fcd8a3a-fd22-4432-9972-4ba06bf0f584"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:36.071793 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.071708    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-credential-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:36.071793 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.071758    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-data\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:36.071793 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.071772    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-workload-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:36.071968 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.071824    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7fcd8a3a-fd22-4432-9972-4ba06bf0f584" (UID: "7fcd8a3a-fd22-4432-9972-4ba06bf0f584"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:36.071968 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.071882    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7fcd8a3a-fd22-4432-9972-4ba06bf0f584" (UID: "7fcd8a3a-fd22-4432-9972-4ba06bf0f584"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:06:36.073381 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.073353    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-token" (OuterVolumeSpecName: "istio-token") pod "7fcd8a3a-fd22-4432-9972-4ba06bf0f584" (UID: "7fcd8a3a-fd22-4432-9972-4ba06bf0f584"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:06:36.073501 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.073390    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7fcd8a3a-fd22-4432-9972-4ba06bf0f584" (UID: "7fcd8a3a-fd22-4432-9972-4ba06bf0f584"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:06:36.073623 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.073606    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-kube-api-access-sggx7" (OuterVolumeSpecName: "kube-api-access-sggx7") pod "7fcd8a3a-fd22-4432-9972-4ba06bf0f584" (UID: "7fcd8a3a-fd22-4432-9972-4ba06bf0f584"). InnerVolumeSpecName "kube-api-access-sggx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:06:36.073623 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.073612    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7fcd8a3a-fd22-4432-9972-4ba06bf0f584" (UID: "7fcd8a3a-fd22-4432-9972-4ba06bf0f584"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:06:36.172543 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.172509    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istiod-ca-cert\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:36.172543 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.172536    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sggx7\" (UniqueName: \"kubernetes.io/projected/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-kube-api-access-sggx7\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:36.172543 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.172546    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-podinfo\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:36.172787 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.172556    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-token\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:36.172787 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.172564    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-istio-envoy\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:36.172787 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.172571    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fcd8a3a-fd22-4432-9972-4ba06bf0f584-workload-certs\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:06:36.760839 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.760801    2580 generic.go:356] "Generic (PLEG): container finished" podID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" containerID="97df18bfc7963e1ad4d4912844fd2795cac3d5233a995a72c64ae6ec80733bc3" exitCode=0
Sep 07 12:06:36.761050 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.760849    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"
Sep 07 12:06:36.761050 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.760892    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx" event={"ID":"7fcd8a3a-fd22-4432-9972-4ba06bf0f584","Type":"ContainerDied","Data":"97df18bfc7963e1ad4d4912844fd2795cac3d5233a995a72c64ae6ec80733bc3"}
Sep 07 12:06:36.761050 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.760955    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx" event={"ID":"7fcd8a3a-fd22-4432-9972-4ba06bf0f584","Type":"ContainerDied","Data":"729fa8629f5a1f1c967b8198740afb552629598d9d1ed98b07b3532768a39cf4"}
Sep 07 12:06:36.761050 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.760984    2580 scope.go:117] "RemoveContainer" containerID="97df18bfc7963e1ad4d4912844fd2795cac3d5233a995a72c64ae6ec80733bc3"
Sep 07 12:06:36.770278 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.770258    2580 scope.go:117] "RemoveContainer" containerID="ce51b1f72dabae3819eaca0bbb27a7c0c550aed59d7358a861913e48b9c08821"
Sep 07 12:06:36.778434 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.778415    2580 scope.go:117] "RemoveContainer" containerID="97df18bfc7963e1ad4d4912844fd2795cac3d5233a995a72c64ae6ec80733bc3"
Sep 07 12:06:36.778745 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:06:36.778724    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97df18bfc7963e1ad4d4912844fd2795cac3d5233a995a72c64ae6ec80733bc3\": container with ID starting with 97df18bfc7963e1ad4d4912844fd2795cac3d5233a995a72c64ae6ec80733bc3 not found: ID does not exist" containerID="97df18bfc7963e1ad4d4912844fd2795cac3d5233a995a72c64ae6ec80733bc3"
Sep 07 12:06:36.778806 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.778755    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97df18bfc7963e1ad4d4912844fd2795cac3d5233a995a72c64ae6ec80733bc3"} err="failed to get container status \"97df18bfc7963e1ad4d4912844fd2795cac3d5233a995a72c64ae6ec80733bc3\": rpc error: code = NotFound desc = could not find container \"97df18bfc7963e1ad4d4912844fd2795cac3d5233a995a72c64ae6ec80733bc3\": container with ID starting with 97df18bfc7963e1ad4d4912844fd2795cac3d5233a995a72c64ae6ec80733bc3 not found: ID does not exist"
Sep 07 12:06:36.778806 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.778773    2580 scope.go:117] "RemoveContainer" containerID="ce51b1f72dabae3819eaca0bbb27a7c0c550aed59d7358a861913e48b9c08821"
Sep 07 12:06:36.779014 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:06:36.778999    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce51b1f72dabae3819eaca0bbb27a7c0c550aed59d7358a861913e48b9c08821\": container with ID starting with ce51b1f72dabae3819eaca0bbb27a7c0c550aed59d7358a861913e48b9c08821 not found: ID does not exist" containerID="ce51b1f72dabae3819eaca0bbb27a7c0c550aed59d7358a861913e48b9c08821"
Sep 07 12:06:36.779058 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.779018    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce51b1f72dabae3819eaca0bbb27a7c0c550aed59d7358a861913e48b9c08821"} err="failed to get container status \"ce51b1f72dabae3819eaca0bbb27a7c0c550aed59d7358a861913e48b9c08821\": rpc error: code = NotFound desc = could not find container \"ce51b1f72dabae3819eaca0bbb27a7c0c550aed59d7358a861913e48b9c08821\": container with ID starting with ce51b1f72dabae3819eaca0bbb27a7c0c550aed59d7358a861913e48b9c08821 not found: ID does not exist"
Sep 07 12:06:36.783709 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.783683    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"]
Sep 07 12:06:36.786628 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:36.786598    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-57104ad7-openshift-default-66555cbf68-sv7tx"]
Sep 07 12:06:38.010352 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:06:38.010311    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" path="/var/lib/kubelet/pods/7fcd8a3a-fd22-4432-9972-4ba06bf0f584/volumes"
Sep 07 12:07:23.763190 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.763105    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs"]
Sep 07 12:07:23.763685 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.763460    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bac6388-971f-449b-893f-d1d1d6dca84d" containerName="payload-processing"
Sep 07 12:07:23.763685 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.763472    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bac6388-971f-449b-893f-d1d1d6dca84d" containerName="payload-processing"
Sep 07 12:07:23.763685 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.763495    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" containerName="istio-proxy"
Sep 07 12:07:23.763685 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.763501    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" containerName="istio-proxy"
Sep 07 12:07:23.763685 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.763508    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" containerName="istio-proxy"
Sep 07 12:07:23.763685 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.763514    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" containerName="istio-proxy"
Sep 07 12:07:23.763685 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.763524    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ef098fb-e205-444a-b97b-cb9a9746130e" containerName="revoke-keys"
Sep 07 12:07:23.763685 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.763529    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ef098fb-e205-444a-b97b-cb9a9746130e" containerName="revoke-keys"
Sep 07 12:07:23.763685 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.763583    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ef098fb-e205-444a-b97b-cb9a9746130e" containerName="revoke-keys"
Sep 07 12:07:23.763685 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.763592    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bac6388-971f-449b-893f-d1d1d6dca84d" containerName="payload-processing"
Sep 07 12:07:23.763685 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.763598    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" containerName="istio-proxy"
Sep 07 12:07:23.763685 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.763604    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fcd8a3a-fd22-4432-9972-4ba06bf0f584" containerName="istio-proxy"
Sep 07 12:07:23.765685 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.765660    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs"
Sep 07 12:07:23.768302 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.768273    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:07:23.768426 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.768349    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:07:23.774463 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.774436    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs"]
Sep 07 12:07:23.817027 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.816986    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5fb59f5-e006-4f2f-b6d9-288ced9d190b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs\" (UID: \"a5fb59f5-e006-4f2f-b6d9-288ced9d190b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs"
Sep 07 12:07:23.918513 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.918471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5fb59f5-e006-4f2f-b6d9-288ced9d190b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs\" (UID: \"a5fb59f5-e006-4f2f-b6d9-288ced9d190b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs"
Sep 07 12:07:23.919104 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:23.919082    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5fb59f5-e006-4f2f-b6d9-288ced9d190b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs\" (UID: \"a5fb59f5-e006-4f2f-b6d9-288ced9d190b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs"
Sep 07 12:07:24.076841 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:24.076746    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs"
Sep 07 12:07:24.353048 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:24.352869    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs"]
Sep 07 12:07:24.355220 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:07:24.355172    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5fb59f5_e006_4f2f_b6d9_288ced9d190b.slice/crio-4edbc058e6e644fdf27ef005c264cfeb423b9741fee42a359b0d6c127fc31881 WatchSource:0}: Error finding container 4edbc058e6e644fdf27ef005c264cfeb423b9741fee42a359b0d6c127fc31881: Status 404 returned error can't find the container with id 4edbc058e6e644fdf27ef005c264cfeb423b9741fee42a359b0d6c127fc31881
Sep 07 12:07:24.999715 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:24.999678    2580 generic.go:356] "Generic (PLEG): container finished" podID="a5fb59f5-e006-4f2f-b6d9-288ced9d190b" containerID="39edbc89e789b1524964eb967c530e9bc99486fe86f024eed29da99f6b12959e" exitCode=0
Sep 07 12:07:25.000132 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:24.999763    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs" event={"ID":"a5fb59f5-e006-4f2f-b6d9-288ced9d190b","Type":"ContainerDied","Data":"39edbc89e789b1524964eb967c530e9bc99486fe86f024eed29da99f6b12959e"}
Sep 07 12:07:25.000132 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:24.999824    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs" event={"ID":"a5fb59f5-e006-4f2f-b6d9-288ced9d190b","Type":"ContainerStarted","Data":"4edbc058e6e644fdf27ef005c264cfeb423b9741fee42a359b0d6c127fc31881"}
Sep 07 12:07:26.142609 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:26.142578    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs"
Sep 07 12:07:26.244008 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:26.243973    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5fb59f5-e006-4f2f-b6d9-288ced9d190b-maas-api-service-ca\") pod \"a5fb59f5-e006-4f2f-b6d9-288ced9d190b\" (UID: \"a5fb59f5-e006-4f2f-b6d9-288ced9d190b\") "
Sep 07 12:07:26.244419 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:26.244391    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5fb59f5-e006-4f2f-b6d9-288ced9d190b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a5fb59f5-e006-4f2f-b6d9-288ced9d190b" (UID: "a5fb59f5-e006-4f2f-b6d9-288ced9d190b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:07:26.345491 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:26.345403    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5fb59f5-e006-4f2f-b6d9-288ced9d190b-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:07:27.013326 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:27.013297    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs"
Sep 07 12:07:27.013503 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:27.013302    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs" event={"ID":"a5fb59f5-e006-4f2f-b6d9-288ced9d190b","Type":"ContainerDied","Data":"4edbc058e6e644fdf27ef005c264cfeb423b9741fee42a359b0d6c127fc31881"}
Sep 07 12:07:27.013503 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:27.013423    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4edbc058e6e644fdf27ef005c264cfeb423b9741fee42a359b0d6c127fc31881"
Sep 07 12:07:34.307341 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:34.307301    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"]
Sep 07 12:07:34.307942 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:34.307636    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85" podUID="e2edce9b-0dde-49b9-8c36-5e1f217bd2ad" containerName="payload-pre-processing" containerID="cri-o://5e3f4b21b93b89bfe3ee2284c63e248d85d09cad12a140ea1e1f77d3055f7131" gracePeriod=30
Sep 07 12:07:34.476735 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:34.476703    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs"]
Sep 07 12:07:34.479064 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:34.479030    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3fa352-r8pgs"]
Sep 07 12:07:34.555059 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:34.555032    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"
Sep 07 12:07:34.621863 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:34.621829    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2edce9b-0dde-49b9-8c36-5e1f217bd2ad-plugins-config-volume\") pod \"e2edce9b-0dde-49b9-8c36-5e1f217bd2ad\" (UID: \"e2edce9b-0dde-49b9-8c36-5e1f217bd2ad\") "
Sep 07 12:07:34.622059 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:34.621931    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97cfx\" (UniqueName: \"kubernetes.io/projected/e2edce9b-0dde-49b9-8c36-5e1f217bd2ad-kube-api-access-97cfx\") pod \"e2edce9b-0dde-49b9-8c36-5e1f217bd2ad\" (UID: \"e2edce9b-0dde-49b9-8c36-5e1f217bd2ad\") "
Sep 07 12:07:34.622248 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:34.622225    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2edce9b-0dde-49b9-8c36-5e1f217bd2ad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e2edce9b-0dde-49b9-8c36-5e1f217bd2ad" (UID: "e2edce9b-0dde-49b9-8c36-5e1f217bd2ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:07:34.624042 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:34.624021    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2edce9b-0dde-49b9-8c36-5e1f217bd2ad-kube-api-access-97cfx" (OuterVolumeSpecName: "kube-api-access-97cfx") pod "e2edce9b-0dde-49b9-8c36-5e1f217bd2ad" (UID: "e2edce9b-0dde-49b9-8c36-5e1f217bd2ad"). InnerVolumeSpecName "kube-api-access-97cfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:07:34.723102 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:34.723064    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97cfx\" (UniqueName: \"kubernetes.io/projected/e2edce9b-0dde-49b9-8c36-5e1f217bd2ad-kube-api-access-97cfx\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:07:34.723102 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:34.723095    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2edce9b-0dde-49b9-8c36-5e1f217bd2ad-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:07:35.054552 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:35.054455    2580 generic.go:356] "Generic (PLEG): container finished" podID="e2edce9b-0dde-49b9-8c36-5e1f217bd2ad" containerID="5e3f4b21b93b89bfe3ee2284c63e248d85d09cad12a140ea1e1f77d3055f7131" exitCode=0
Sep 07 12:07:35.054552 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:35.054503    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"
Sep 07 12:07:35.054552 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:35.054523    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85" event={"ID":"e2edce9b-0dde-49b9-8c36-5e1f217bd2ad","Type":"ContainerDied","Data":"5e3f4b21b93b89bfe3ee2284c63e248d85d09cad12a140ea1e1f77d3055f7131"}
Sep 07 12:07:35.054825 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:35.054578    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85" event={"ID":"e2edce9b-0dde-49b9-8c36-5e1f217bd2ad","Type":"ContainerDied","Data":"3e4cf7c700f32558d4146b09cf26f47fcb943aa2a293186a5a679d9d3c348ed0"}
Sep 07 12:07:35.054825 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:35.054606    2580 scope.go:117] "RemoveContainer" containerID="5e3f4b21b93b89bfe3ee2284c63e248d85d09cad12a140ea1e1f77d3055f7131"
Sep 07 12:07:35.063442 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:35.063420    2580 scope.go:117] "RemoveContainer" containerID="5e3f4b21b93b89bfe3ee2284c63e248d85d09cad12a140ea1e1f77d3055f7131"
Sep 07 12:07:35.063769 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:07:35.063748    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e3f4b21b93b89bfe3ee2284c63e248d85d09cad12a140ea1e1f77d3055f7131\": container with ID starting with 5e3f4b21b93b89bfe3ee2284c63e248d85d09cad12a140ea1e1f77d3055f7131 not found: ID does not exist" containerID="5e3f4b21b93b89bfe3ee2284c63e248d85d09cad12a140ea1e1f77d3055f7131"
Sep 07 12:07:35.063812 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:35.063779    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e3f4b21b93b89bfe3ee2284c63e248d85d09cad12a140ea1e1f77d3055f7131"} err="failed to get container status \"5e3f4b21b93b89bfe3ee2284c63e248d85d09cad12a140ea1e1f77d3055f7131\": rpc error: code = NotFound desc = could not find container \"5e3f4b21b93b89bfe3ee2284c63e248d85d09cad12a140ea1e1f77d3055f7131\": container with ID starting with 5e3f4b21b93b89bfe3ee2284c63e248d85d09cad12a140ea1e1f77d3055f7131 not found: ID does not exist"
Sep 07 12:07:35.075945 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:35.075915    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"]
Sep 07 12:07:35.079125 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:35.079098    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-3fa352-6b95c5667d-jhj85"]
Sep 07 12:07:36.010256 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:36.010224    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5fb59f5-e006-4f2f-b6d9-288ced9d190b" path="/var/lib/kubelet/pods/a5fb59f5-e006-4f2f-b6d9-288ced9d190b/volumes"
Sep 07 12:07:36.010635 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:36.010608    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2edce9b-0dde-49b9-8c36-5e1f217bd2ad" path="/var/lib/kubelet/pods/e2edce9b-0dde-49b9-8c36-5e1f217bd2ad/volumes"
Sep 07 12:07:41.937127 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:41.937078    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"]
Sep 07 12:07:41.937539 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:41.937456    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5fb59f5-e006-4f2f-b6d9-288ced9d190b" containerName="revoke-keys"
Sep 07 12:07:41.937539 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:41.937472    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5fb59f5-e006-4f2f-b6d9-288ced9d190b" containerName="revoke-keys"
Sep 07 12:07:41.937539 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:41.937493    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2edce9b-0dde-49b9-8c36-5e1f217bd2ad" containerName="payload-pre-processing"
Sep 07 12:07:41.937539 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:41.937502    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2edce9b-0dde-49b9-8c36-5e1f217bd2ad" containerName="payload-pre-processing"
Sep 07 12:07:41.937690 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:41.937598    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5fb59f5-e006-4f2f-b6d9-288ced9d190b" containerName="revoke-keys"
Sep 07 12:07:41.937690 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:41.937611    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2edce9b-0dde-49b9-8c36-5e1f217bd2ad" containerName="payload-pre-processing"
Sep 07 12:07:41.940561 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:41.940536    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:41.943073 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:41.943050    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-cae4d5-openshift-default-dockercfg-b9b7q\""
Sep 07 12:07:41.951748 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:41.951717    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"]
Sep 07 12:07:42.091028 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.090990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b29f25b-404f-4090-b430-d3417cb6394b-istio-token\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.091028 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.091034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-istio-envoy\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.091295 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.091060    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-workload-certs\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.091295 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.091152    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b29f25b-404f-4090-b430-d3417cb6394b-istio-podinfo\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.091295 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.091183    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-workload-socket\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.091295 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.091240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b29f25b-404f-4090-b430-d3417cb6394b-istiod-ca-cert\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.091295 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.091275    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-credential-socket\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.091461 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.091325    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgzq2\" (UniqueName: \"kubernetes.io/projected/9b29f25b-404f-4090-b430-d3417cb6394b-kube-api-access-tgzq2\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.091461 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.091345    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-istio-data\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.191878 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.191780    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b29f25b-404f-4090-b430-d3417cb6394b-istio-podinfo\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.191878 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.191822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-workload-socket\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.191878 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.191850    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b29f25b-404f-4090-b430-d3417cb6394b-istiod-ca-cert\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.191878 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.191876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-credential-socket\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.192232 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.191898    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgzq2\" (UniqueName: \"kubernetes.io/projected/9b29f25b-404f-4090-b430-d3417cb6394b-kube-api-access-tgzq2\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.192232 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.191916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-istio-data\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.192232 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.191961    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b29f25b-404f-4090-b430-d3417cb6394b-istio-token\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.192232 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.191991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-istio-envoy\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.192232 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.192017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-workload-certs\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.192498 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.192352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-workload-socket\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.192498 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.192436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-credential-socket\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.192498 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.192479    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-workload-certs\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.192726 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.192704    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-istio-data\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.192795 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.192726    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b29f25b-404f-4090-b430-d3417cb6394b-istiod-ca-cert\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.194409 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.194388    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b29f25b-404f-4090-b430-d3417cb6394b-istio-podinfo\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.194525 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.194505    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-istio-envoy\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.199817 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.199780    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b29f25b-404f-4090-b430-d3417cb6394b-istio-token\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.199927 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.199911    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgzq2\" (UniqueName: \"kubernetes.io/projected/9b29f25b-404f-4090-b430-d3417cb6394b-kube-api-access-tgzq2\") pod \"e2e-noten-cae4d5-openshift-default-67946bf976-txmvn\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") " pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.252828 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.252787    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:42.536010 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.535982    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"]
Sep 07 12:07:42.538384 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:07:42.538357    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b29f25b_404f_4090_b430_d3417cb6394b.slice/crio-151f6c660542b08162c80b22fde15616c0ae0b2349cd9dc71d508c66cba981e0 WatchSource:0}: Error finding container 151f6c660542b08162c80b22fde15616c0ae0b2349cd9dc71d508c66cba981e0: Status 404 returned error can't find the container with id 151f6c660542b08162c80b22fde15616c0ae0b2349cd9dc71d508c66cba981e0
Sep 07 12:07:42.541034 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.541000    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:07:42.541129 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.541084    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:07:42.541212 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:42.541129    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:07:43.099876 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:43.099838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn" event={"ID":"9b29f25b-404f-4090-b430-d3417cb6394b","Type":"ContainerStarted","Data":"48ce482ff48eb78ac90ede32358cfc4a2fd609d45b46c0d3ac1d67247f854cc1"}
Sep 07 12:07:43.100297 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:43.099899    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn" event={"ID":"9b29f25b-404f-4090-b430-d3417cb6394b","Type":"ContainerStarted","Data":"151f6c660542b08162c80b22fde15616c0ae0b2349cd9dc71d508c66cba981e0"}
Sep 07 12:07:43.120744 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:43.120684    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn" podStartSLOduration=2.120666529 podStartE2EDuration="2.120666529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:07:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:07:43.118543657 +0000 UTC m=+1217.614433080" watchObservedRunningTime="2026-09-07 12:07:43.120666529 +0000 UTC m=+1217.616555948"
Sep 07 12:07:43.253542 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:43.253510    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:43.258970 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:43.258940    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:44.104818 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:44.104783    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:44.105813 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:44.105793    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:07:59.373627 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:59.373588    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"]
Sep 07 12:07:59.374137 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:07:59.373835    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn" podUID="9b29f25b-404f-4090-b430-d3417cb6394b" containerName="istio-proxy" containerID="cri-o://48ce482ff48eb78ac90ede32358cfc4a2fd609d45b46c0d3ac1d67247f854cc1" gracePeriod=30
Sep 07 12:08:04.625138 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.625106    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:08:04.704038 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.703950    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b29f25b-404f-4090-b430-d3417cb6394b-istiod-ca-cert\") pod \"9b29f25b-404f-4090-b430-d3417cb6394b\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") "
Sep 07 12:08:04.704038 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704004    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-istio-data\") pod \"9b29f25b-404f-4090-b430-d3417cb6394b\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") "
Sep 07 12:08:04.704038 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704030    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-workload-socket\") pod \"9b29f25b-404f-4090-b430-d3417cb6394b\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") "
Sep 07 12:08:04.704321 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704152    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-credential-socket\") pod \"9b29f25b-404f-4090-b430-d3417cb6394b\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") "
Sep 07 12:08:04.704321 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704265    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b29f25b-404f-4090-b430-d3417cb6394b-istio-token\") pod \"9b29f25b-404f-4090-b430-d3417cb6394b\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") "
Sep 07 12:08:04.704321 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704294    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-workload-certs\") pod \"9b29f25b-404f-4090-b430-d3417cb6394b\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") "
Sep 07 12:08:04.704466 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704342    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-istio-envoy\") pod \"9b29f25b-404f-4090-b430-d3417cb6394b\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") "
Sep 07 12:08:04.704466 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704377    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgzq2\" (UniqueName: \"kubernetes.io/projected/9b29f25b-404f-4090-b430-d3417cb6394b-kube-api-access-tgzq2\") pod \"9b29f25b-404f-4090-b430-d3417cb6394b\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") "
Sep 07 12:08:04.704466 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704338    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9b29f25b-404f-4090-b430-d3417cb6394b" (UID: "9b29f25b-404f-4090-b430-d3417cb6394b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:04.704466 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704423    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b29f25b-404f-4090-b430-d3417cb6394b-istio-podinfo\") pod \"9b29f25b-404f-4090-b430-d3417cb6394b\" (UID: \"9b29f25b-404f-4090-b430-d3417cb6394b\") "
Sep 07 12:08:04.704466 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704348    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-istio-data" (OuterVolumeSpecName: "istio-data") pod "9b29f25b-404f-4090-b430-d3417cb6394b" (UID: "9b29f25b-404f-4090-b430-d3417cb6394b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:04.704466 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704448    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9b29f25b-404f-4090-b430-d3417cb6394b" (UID: "9b29f25b-404f-4090-b430-d3417cb6394b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:04.704466 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704462    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b29f25b-404f-4090-b430-d3417cb6394b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9b29f25b-404f-4090-b430-d3417cb6394b" (UID: "9b29f25b-404f-4090-b430-d3417cb6394b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:08:04.704792 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704609    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9b29f25b-404f-4090-b430-d3417cb6394b" (UID: "9b29f25b-404f-4090-b430-d3417cb6394b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:04.704792 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704685    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-workload-certs\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:04.704792 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704709    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b29f25b-404f-4090-b430-d3417cb6394b-istiod-ca-cert\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:04.704792 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704722    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-istio-data\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:04.704792 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704734    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-workload-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:04.704792 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.704747    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-credential-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:04.706501 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.706481    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9b29f25b-404f-4090-b430-d3417cb6394b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9b29f25b-404f-4090-b430-d3417cb6394b" (UID: "9b29f25b-404f-4090-b430-d3417cb6394b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:08:04.706620 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.706554    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b29f25b-404f-4090-b430-d3417cb6394b-kube-api-access-tgzq2" (OuterVolumeSpecName: "kube-api-access-tgzq2") pod "9b29f25b-404f-4090-b430-d3417cb6394b" (UID: "9b29f25b-404f-4090-b430-d3417cb6394b"). InnerVolumeSpecName "kube-api-access-tgzq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:08:04.706831 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.706814    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9b29f25b-404f-4090-b430-d3417cb6394b" (UID: "9b29f25b-404f-4090-b430-d3417cb6394b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:04.706897 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.706828    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b29f25b-404f-4090-b430-d3417cb6394b-istio-token" (OuterVolumeSpecName: "istio-token") pod "9b29f25b-404f-4090-b430-d3417cb6394b" (UID: "9b29f25b-404f-4090-b430-d3417cb6394b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:08:04.805734 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.805685    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b29f25b-404f-4090-b430-d3417cb6394b-istio-token\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:04.805734 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.805727    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b29f25b-404f-4090-b430-d3417cb6394b-istio-envoy\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:04.805734 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.805740    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgzq2\" (UniqueName: \"kubernetes.io/projected/9b29f25b-404f-4090-b430-d3417cb6394b-kube-api-access-tgzq2\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:04.805734 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:04.805749    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b29f25b-404f-4090-b430-d3417cb6394b-istio-podinfo\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:05.212292 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:05.212252    2580 generic.go:356] "Generic (PLEG): container finished" podID="9b29f25b-404f-4090-b430-d3417cb6394b" containerID="48ce482ff48eb78ac90ede32358cfc4a2fd609d45b46c0d3ac1d67247f854cc1" exitCode=0
Sep 07 12:08:05.212483 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:05.212302    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"
Sep 07 12:08:05.212483 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:05.212334    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn" event={"ID":"9b29f25b-404f-4090-b430-d3417cb6394b","Type":"ContainerDied","Data":"48ce482ff48eb78ac90ede32358cfc4a2fd609d45b46c0d3ac1d67247f854cc1"}
Sep 07 12:08:05.212483 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:05.212388    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn" event={"ID":"9b29f25b-404f-4090-b430-d3417cb6394b","Type":"ContainerDied","Data":"151f6c660542b08162c80b22fde15616c0ae0b2349cd9dc71d508c66cba981e0"}
Sep 07 12:08:05.212483 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:05.212411    2580 scope.go:117] "RemoveContainer" containerID="48ce482ff48eb78ac90ede32358cfc4a2fd609d45b46c0d3ac1d67247f854cc1"
Sep 07 12:08:05.222221 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:05.222185    2580 scope.go:117] "RemoveContainer" containerID="48ce482ff48eb78ac90ede32358cfc4a2fd609d45b46c0d3ac1d67247f854cc1"
Sep 07 12:08:05.222537 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:08:05.222513    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48ce482ff48eb78ac90ede32358cfc4a2fd609d45b46c0d3ac1d67247f854cc1\": container with ID starting with 48ce482ff48eb78ac90ede32358cfc4a2fd609d45b46c0d3ac1d67247f854cc1 not found: ID does not exist" containerID="48ce482ff48eb78ac90ede32358cfc4a2fd609d45b46c0d3ac1d67247f854cc1"
Sep 07 12:08:05.222628 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:05.222544    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48ce482ff48eb78ac90ede32358cfc4a2fd609d45b46c0d3ac1d67247f854cc1"} err="failed to get container status \"48ce482ff48eb78ac90ede32358cfc4a2fd609d45b46c0d3ac1d67247f854cc1\": rpc error: code = NotFound desc = could not find container \"48ce482ff48eb78ac90ede32358cfc4a2fd609d45b46c0d3ac1d67247f854cc1\": container with ID starting with 48ce482ff48eb78ac90ede32358cfc4a2fd609d45b46c0d3ac1d67247f854cc1 not found: ID does not exist"
Sep 07 12:08:05.233875 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:05.233845    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"]
Sep 07 12:08:05.237381 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:05.237356    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-cae4d5-openshift-default-67946bf976-txmvn"]
Sep 07 12:08:06.010365 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.010330    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b29f25b-404f-4090-b430-d3417cb6394b" path="/var/lib/kubelet/pods/9b29f25b-404f-4090-b430-d3417cb6394b/volumes"
Sep 07 12:08:06.217539 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.217504    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"]
Sep 07 12:08:06.218310 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.218284    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b29f25b-404f-4090-b430-d3417cb6394b" containerName="istio-proxy"
Sep 07 12:08:06.218480 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.218469    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b29f25b-404f-4090-b430-d3417cb6394b" containerName="istio-proxy"
Sep 07 12:08:06.218651 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.218639    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b29f25b-404f-4090-b430-d3417cb6394b" containerName="istio-proxy"
Sep 07 12:08:06.222671 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.222641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.228848 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.228822    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5f62f6f8-openshift-default-dockercfg-zbwgj\""
Sep 07 12:08:06.243684 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.243647    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"]
Sep 07 12:08:06.321304 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.321186    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-podinfo\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.321486 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.321344    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istiod-ca-cert\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.321486 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.321375    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-envoy\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.321486 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.321408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r6kx\" (UniqueName: \"kubernetes.io/projected/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-kube-api-access-9r6kx\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.321486 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.321454    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-token\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.321738 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.321488    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-credential-socket\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.321738 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.321528    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-workload-certs\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.321738 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.321555    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-workload-socket\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.321738 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.321579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-data\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.422651 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.422607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-podinfo\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.422848 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.422696    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istiod-ca-cert\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.422848 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.422719    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-envoy\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.422848 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.422738    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r6kx\" (UniqueName: \"kubernetes.io/projected/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-kube-api-access-9r6kx\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.422848 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.422768    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-token\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.422848 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.422799    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-credential-socket\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.422848 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.422824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-workload-certs\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.422848 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.422849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-workload-socket\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.423233 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.422873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-data\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.423311 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.423240    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-credential-socket\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.423311 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.423248    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-workload-certs\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.423419 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.423339    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-data\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.423419 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.423404    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-workload-socket\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.423589 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.423571    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istiod-ca-cert\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.425078 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.425047    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-envoy\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.425337 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.425319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-podinfo\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.430439 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.430409    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-token\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.430584 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.430565    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r6kx\" (UniqueName: \"kubernetes.io/projected/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-kube-api-access-9r6kx\") pod \"e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") " pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.534399 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.534360    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:06.811425 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.811395    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"]
Sep 07 12:08:06.814027 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:08:06.813990    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b5bb7df_eff7_4a64_9c3a_89e2cd38fadb.slice/crio-ee54c925339a8911973b55f929ec035f51dc7f916c5033a0fe6396f59599e18e WatchSource:0}: Error finding container ee54c925339a8911973b55f929ec035f51dc7f916c5033a0fe6396f59599e18e: Status 404 returned error can't find the container with id ee54c925339a8911973b55f929ec035f51dc7f916c5033a0fe6396f59599e18e
Sep 07 12:08:06.816864 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.816834    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:08:06.816941 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.816905    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:08:06.816941 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:06.816934    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:08:07.228580 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:07.228545    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks" event={"ID":"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb","Type":"ContainerStarted","Data":"48051f2ec6e722adb4f95db13bdf9fb1ee140b85b31b65a92519ed86df26c1c6"}
Sep 07 12:08:07.228957 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:07.228600    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks" event={"ID":"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb","Type":"ContainerStarted","Data":"ee54c925339a8911973b55f929ec035f51dc7f916c5033a0fe6396f59599e18e"}
Sep 07 12:08:07.250111 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:07.250061    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks" podStartSLOduration=1.250044933 podStartE2EDuration="1.250044933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:08:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:08:07.248616512 +0000 UTC m=+1241.744505937" watchObservedRunningTime="2026-09-07 12:08:07.250044933 +0000 UTC m=+1241.745934355"
Sep 07 12:08:07.535672 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:07.535569    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:07.540792 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:07.540763    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:08.233844 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:08.233815    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:08.234876 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:08.234859    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:10.879483 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:10.879446    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"]
Sep 07 12:08:10.882347 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:10.882326    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"
Sep 07 12:08:10.885113 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:10.885088    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-5f62f6f8-dockercfg-w7qt2\""
Sep 07 12:08:10.885284 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:10.885132    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-5f62f6f8\""
Sep 07 12:08:10.896127 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:10.896099    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"]
Sep 07 12:08:10.964913 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:10.964592    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbz4n\" (UniqueName: \"kubernetes.io/projected/b133963b-1833-4846-a668-f50a6f124cae-kube-api-access-sbz4n\") pod \"payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24\" (UID: \"b133963b-1833-4846-a668-f50a6f124cae\") " pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"
Sep 07 12:08:10.964913 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:10.964842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b133963b-1833-4846-a668-f50a6f124cae-plugins-config-volume\") pod \"payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24\" (UID: \"b133963b-1833-4846-a668-f50a6f124cae\") " pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"
Sep 07 12:08:11.066497 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:11.066447    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbz4n\" (UniqueName: \"kubernetes.io/projected/b133963b-1833-4846-a668-f50a6f124cae-kube-api-access-sbz4n\") pod \"payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24\" (UID: \"b133963b-1833-4846-a668-f50a6f124cae\") " pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"
Sep 07 12:08:11.066709 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:11.066574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b133963b-1833-4846-a668-f50a6f124cae-plugins-config-volume\") pod \"payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24\" (UID: \"b133963b-1833-4846-a668-f50a6f124cae\") " pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"
Sep 07 12:08:11.067468 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:11.067433    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b133963b-1833-4846-a668-f50a6f124cae-plugins-config-volume\") pod \"payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24\" (UID: \"b133963b-1833-4846-a668-f50a6f124cae\") " pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"
Sep 07 12:08:11.074705 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:11.074668    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbz4n\" (UniqueName: \"kubernetes.io/projected/b133963b-1833-4846-a668-f50a6f124cae-kube-api-access-sbz4n\") pod \"payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24\" (UID: \"b133963b-1833-4846-a668-f50a6f124cae\") " pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"
Sep 07 12:08:11.194488 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:11.194402    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"
Sep 07 12:08:11.464379 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:11.464354    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"]
Sep 07 12:08:11.466414 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:08:11.466386    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb133963b_1833_4846_a668_f50a6f124cae.slice/crio-f2d5edd12ca191df5516b80fe8ba66bcb6ff3f9b356d7b35dba62935869239cf WatchSource:0}: Error finding container f2d5edd12ca191df5516b80fe8ba66bcb6ff3f9b356d7b35dba62935869239cf: Status 404 returned error can't find the container with id f2d5edd12ca191df5516b80fe8ba66bcb6ff3f9b356d7b35dba62935869239cf
Sep 07 12:08:12.257267 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:12.257232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24" event={"ID":"b133963b-1833-4846-a668-f50a6f124cae","Type":"ContainerStarted","Data":"8e8023624cd775da5680af0dd63dc708398494405dcd7ed9542f1a0b0453794a"}
Sep 07 12:08:12.257730 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:12.257295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24" event={"ID":"b133963b-1833-4846-a668-f50a6f124cae","Type":"ContainerStarted","Data":"f2d5edd12ca191df5516b80fe8ba66bcb6ff3f9b356d7b35dba62935869239cf"}
Sep 07 12:08:12.257730 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:12.257323    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"
Sep 07 12:08:12.273371 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:12.273324    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24" podStartSLOduration=2.273306532 podStartE2EDuration="2.273306532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:08:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:08:12.271653801 +0000 UTC m=+1246.767543225" watchObservedRunningTime="2026-09-07 12:08:12.273306532 +0000 UTC m=+1246.769195952"
Sep 07 12:08:23.265915 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:23.265820    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"
Sep 07 12:08:25.751012 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:25.750920    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6"]
Sep 07 12:08:25.756957 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:25.756932    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6"
Sep 07 12:08:25.760032 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:25.759699    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:08:25.760032 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:25.759710    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:08:25.761317 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:25.761292    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6"]
Sep 07 12:08:25.808568 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:25.808533    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6\" (UID: \"8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6"
Sep 07 12:08:25.909647 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:25.909602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6\" (UID: \"8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6"
Sep 07 12:08:25.912150 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:25.912111    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:08:25.923333 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:25.923293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6\" (UID: \"8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6"
Sep 07 12:08:26.074021 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:26.073981    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:08:26.082289 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:26.082256    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6"
Sep 07 12:08:26.356937 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:26.356908    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6"]
Sep 07 12:08:26.358934 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:08:26.358906    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c8d15e6_c7e1_43aa_bfe1_bd11b4d53969.slice/crio-a2cd72231f0cdbd7c564bd0b672888981c850ada49de2b2bdc4c2182aa36a036 WatchSource:0}: Error finding container a2cd72231f0cdbd7c564bd0b672888981c850ada49de2b2bdc4c2182aa36a036: Status 404 returned error can't find the container with id a2cd72231f0cdbd7c564bd0b672888981c850ada49de2b2bdc4c2182aa36a036
Sep 07 12:08:27.336627 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:27.336538    2580 generic.go:356] "Generic (PLEG): container finished" podID="8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969" containerID="209ec40e60657148539d1a07c784f62f8a52f7a2a6289e83f6223d9561d9305a" exitCode=0
Sep 07 12:08:27.337008 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:27.336631    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6" event={"ID":"8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969","Type":"ContainerDied","Data":"209ec40e60657148539d1a07c784f62f8a52f7a2a6289e83f6223d9561d9305a"}
Sep 07 12:08:27.337008 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:27.336701    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6" event={"ID":"8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969","Type":"ContainerStarted","Data":"a2cd72231f0cdbd7c564bd0b672888981c850ada49de2b2bdc4c2182aa36a036"}
Sep 07 12:08:28.485838 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:28.485810    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6"
Sep 07 12:08:28.641870 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:28.641833    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969-maas-api-service-ca\") pod \"8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969\" (UID: \"8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969\") "
Sep 07 12:08:28.642231 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:28.642180    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969" (UID: "8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:08:28.743048 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:28.743003    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:29.348622 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:29.348582    2580 generic.go:356] "Generic (PLEG): container finished" podID="5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" containerID="48051f2ec6e722adb4f95db13bdf9fb1ee140b85b31b65a92519ed86df26c1c6" exitCode=0
Sep 07 12:08:29.348814 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:29.348657    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks" event={"ID":"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb","Type":"ContainerDied","Data":"48051f2ec6e722adb4f95db13bdf9fb1ee140b85b31b65a92519ed86df26c1c6"}
Sep 07 12:08:29.349135 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:29.349115    2580 scope.go:117] "RemoveContainer" containerID="48051f2ec6e722adb4f95db13bdf9fb1ee140b85b31b65a92519ed86df26c1c6"
Sep 07 12:08:29.350146 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:29.350126    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6" event={"ID":"8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969","Type":"ContainerDied","Data":"a2cd72231f0cdbd7c564bd0b672888981c850ada49de2b2bdc4c2182aa36a036"}
Sep 07 12:08:29.350275 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:29.350150    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6"
Sep 07 12:08:29.350275 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:29.350162    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2cd72231f0cdbd7c564bd0b672888981c850ada49de2b2bdc4c2182aa36a036"
Sep 07 12:08:29.350384 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:29.350284    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:08:29.350384 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:29.350345    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:08:29.350384 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:29.350376    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:08:30.361327 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:30.361287    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks" event={"ID":"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb","Type":"ContainerStarted","Data":"64edd28da7c5dfe4add69b0b129ff828837db6fb6cc71f0885fba8b4eb87e296"}
Sep 07 12:08:30.535103 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:30.535060    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:30.535342 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:30.535320    2580 patch_prober.go:28] interesting pod/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 07 12:08:30.535391 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:30.535369    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks" podUID="5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 07 12:08:31.535540 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:31.535502    2580 patch_prober.go:28] interesting pod/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 07 12:08:31.535946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:31.535563    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks" podUID="5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 07 12:08:32.535819 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:32.535782    2580 patch_prober.go:28] interesting pod/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 07 12:08:32.536237 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:32.535844    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks" podUID="5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 07 12:08:33.535739 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:33.535705    2580 patch_prober.go:28] interesting pod/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 07 12:08:33.535927 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:33.535759    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks" podUID="5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 07 12:08:34.535613 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:34.535572    2580 patch_prober.go:28] interesting pod/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 07 12:08:34.535817 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:34.535643    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks" podUID="5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 07 12:08:35.535749 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:35.535710    2580 patch_prober.go:28] interesting pod/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 07 12:08:35.536160 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:35.535768    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks" podUID="5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 07 12:08:36.510093 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.510059    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"]
Sep 07 12:08:36.511101 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.511022    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h" podUID="e5eb1e62-13b1-4973-aa34-ff4839fea413" containerName="payload-processing" containerID="cri-o://66e298d5dc84e36310f4acd484aad0d1af1b6d4c093a49826bcb137f09691f3b" gracePeriod=30
Sep 07 12:08:36.525896 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.525860    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"]
Sep 07 12:08:36.526401 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.526349    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv" podUID="88619833-32c5-4418-8925-91d78f45518a" containerName="payload-pre-processing" containerID="cri-o://9300d3a7be7e46773721a045ddf4fb67feaffe315d9bfd2aa946f5e45498416c" gracePeriod=30
Sep 07 12:08:36.542058 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.542009    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:36.542864 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.542658    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:36.543544 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.543527    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:08:36.687507 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.687441    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6"]
Sep 07 12:08:36.690271 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.690136    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-c7d751-9ngm6"]
Sep 07 12:08:36.776969 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.776944    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"
Sep 07 12:08:36.809066 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.809042    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"
Sep 07 12:08:36.924869 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.924830    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tt2d\" (UniqueName: \"kubernetes.io/projected/e5eb1e62-13b1-4973-aa34-ff4839fea413-kube-api-access-7tt2d\") pod \"e5eb1e62-13b1-4973-aa34-ff4839fea413\" (UID: \"e5eb1e62-13b1-4973-aa34-ff4839fea413\") "
Sep 07 12:08:36.925071 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.924882    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5eb1e62-13b1-4973-aa34-ff4839fea413-plugins-config-volume\") pod \"e5eb1e62-13b1-4973-aa34-ff4839fea413\" (UID: \"e5eb1e62-13b1-4973-aa34-ff4839fea413\") "
Sep 07 12:08:36.925071 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.924928    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpvrz\" (UniqueName: \"kubernetes.io/projected/88619833-32c5-4418-8925-91d78f45518a-kube-api-access-xpvrz\") pod \"88619833-32c5-4418-8925-91d78f45518a\" (UID: \"88619833-32c5-4418-8925-91d78f45518a\") "
Sep 07 12:08:36.925071 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.924969    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88619833-32c5-4418-8925-91d78f45518a-plugins-config-volume\") pod \"88619833-32c5-4418-8925-91d78f45518a\" (UID: \"88619833-32c5-4418-8925-91d78f45518a\") "
Sep 07 12:08:36.925373 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.925349    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5eb1e62-13b1-4973-aa34-ff4839fea413-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e5eb1e62-13b1-4973-aa34-ff4839fea413" (UID: "e5eb1e62-13b1-4973-aa34-ff4839fea413"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:08:36.925373 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.925362    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88619833-32c5-4418-8925-91d78f45518a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "88619833-32c5-4418-8925-91d78f45518a" (UID: "88619833-32c5-4418-8925-91d78f45518a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:08:36.926999 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.926977    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5eb1e62-13b1-4973-aa34-ff4839fea413-kube-api-access-7tt2d" (OuterVolumeSpecName: "kube-api-access-7tt2d") pod "e5eb1e62-13b1-4973-aa34-ff4839fea413" (UID: "e5eb1e62-13b1-4973-aa34-ff4839fea413"). InnerVolumeSpecName "kube-api-access-7tt2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:08:36.927062 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:36.926994    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88619833-32c5-4418-8925-91d78f45518a-kube-api-access-xpvrz" (OuterVolumeSpecName: "kube-api-access-xpvrz") pod "88619833-32c5-4418-8925-91d78f45518a" (UID: "88619833-32c5-4418-8925-91d78f45518a"). InnerVolumeSpecName "kube-api-access-xpvrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:08:37.025733 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.025634    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tt2d\" (UniqueName: \"kubernetes.io/projected/e5eb1e62-13b1-4973-aa34-ff4839fea413-kube-api-access-7tt2d\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:37.025733 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.025674    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5eb1e62-13b1-4973-aa34-ff4839fea413-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:37.025733 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.025683    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpvrz\" (UniqueName: \"kubernetes.io/projected/88619833-32c5-4418-8925-91d78f45518a-kube-api-access-xpvrz\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:37.025733 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.025693    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88619833-32c5-4418-8925-91d78f45518a-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:37.403521 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.403487    2580 generic.go:356] "Generic (PLEG): container finished" podID="e5eb1e62-13b1-4973-aa34-ff4839fea413" containerID="66e298d5dc84e36310f4acd484aad0d1af1b6d4c093a49826bcb137f09691f3b" exitCode=0
Sep 07 12:08:37.403839 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.403543    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"
Sep 07 12:08:37.403839 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.403567    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h" event={"ID":"e5eb1e62-13b1-4973-aa34-ff4839fea413","Type":"ContainerDied","Data":"66e298d5dc84e36310f4acd484aad0d1af1b6d4c093a49826bcb137f09691f3b"}
Sep 07 12:08:37.403839 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.403609    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h" event={"ID":"e5eb1e62-13b1-4973-aa34-ff4839fea413","Type":"ContainerDied","Data":"65589822d77a0257c70e1b22e40f40f229b881d60d66bb0954e33b51db1f2a5e"}
Sep 07 12:08:37.403839 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.403638    2580 scope.go:117] "RemoveContainer" containerID="66e298d5dc84e36310f4acd484aad0d1af1b6d4c093a49826bcb137f09691f3b"
Sep 07 12:08:37.404902 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.404876    2580 generic.go:356] "Generic (PLEG): container finished" podID="88619833-32c5-4418-8925-91d78f45518a" containerID="9300d3a7be7e46773721a045ddf4fb67feaffe315d9bfd2aa946f5e45498416c" exitCode=0
Sep 07 12:08:37.404977 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.404927    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"
Sep 07 12:08:37.405036 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.404964    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv" event={"ID":"88619833-32c5-4418-8925-91d78f45518a","Type":"ContainerDied","Data":"9300d3a7be7e46773721a045ddf4fb67feaffe315d9bfd2aa946f5e45498416c"}
Sep 07 12:08:37.405036 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.405017    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv" event={"ID":"88619833-32c5-4418-8925-91d78f45518a","Type":"ContainerDied","Data":"b10c220ae08451d4b32587d31ca6c448315ce63e24e0dfc7caff0f329467cdea"}
Sep 07 12:08:37.413396 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.413378    2580 scope.go:117] "RemoveContainer" containerID="66e298d5dc84e36310f4acd484aad0d1af1b6d4c093a49826bcb137f09691f3b"
Sep 07 12:08:37.413749 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:08:37.413730    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66e298d5dc84e36310f4acd484aad0d1af1b6d4c093a49826bcb137f09691f3b\": container with ID starting with 66e298d5dc84e36310f4acd484aad0d1af1b6d4c093a49826bcb137f09691f3b not found: ID does not exist" containerID="66e298d5dc84e36310f4acd484aad0d1af1b6d4c093a49826bcb137f09691f3b"
Sep 07 12:08:37.413805 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.413757    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66e298d5dc84e36310f4acd484aad0d1af1b6d4c093a49826bcb137f09691f3b"} err="failed to get container status \"66e298d5dc84e36310f4acd484aad0d1af1b6d4c093a49826bcb137f09691f3b\": rpc error: code = NotFound desc = could not find container \"66e298d5dc84e36310f4acd484aad0d1af1b6d4c093a49826bcb137f09691f3b\": container with ID starting with 66e298d5dc84e36310f4acd484aad0d1af1b6d4c093a49826bcb137f09691f3b not found: ID does not exist"
Sep 07 12:08:37.413805 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.413775    2580 scope.go:117] "RemoveContainer" containerID="9300d3a7be7e46773721a045ddf4fb67feaffe315d9bfd2aa946f5e45498416c"
Sep 07 12:08:37.421991 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.421968    2580 scope.go:117] "RemoveContainer" containerID="9300d3a7be7e46773721a045ddf4fb67feaffe315d9bfd2aa946f5e45498416c"
Sep 07 12:08:37.422454 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:08:37.422434    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9300d3a7be7e46773721a045ddf4fb67feaffe315d9bfd2aa946f5e45498416c\": container with ID starting with 9300d3a7be7e46773721a045ddf4fb67feaffe315d9bfd2aa946f5e45498416c not found: ID does not exist" containerID="9300d3a7be7e46773721a045ddf4fb67feaffe315d9bfd2aa946f5e45498416c"
Sep 07 12:08:37.422516 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.422461    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9300d3a7be7e46773721a045ddf4fb67feaffe315d9bfd2aa946f5e45498416c"} err="failed to get container status \"9300d3a7be7e46773721a045ddf4fb67feaffe315d9bfd2aa946f5e45498416c\": rpc error: code = NotFound desc = could not find container \"9300d3a7be7e46773721a045ddf4fb67feaffe315d9bfd2aa946f5e45498416c\": container with ID starting with 9300d3a7be7e46773721a045ddf4fb67feaffe315d9bfd2aa946f5e45498416c not found: ID does not exist"
Sep 07 12:08:37.424987 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.424964    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"]
Sep 07 12:08:37.434064 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.434037    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-c7d751-b79f5d5b7-rtm4h"]
Sep 07 12:08:37.448551 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.448517    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"]
Sep 07 12:08:37.453028 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:37.452999    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-c7d751-68f7654c7d-zs5tv"]
Sep 07 12:08:38.010567 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:38.010533    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88619833-32c5-4418-8925-91d78f45518a" path="/var/lib/kubelet/pods/88619833-32c5-4418-8925-91d78f45518a/volumes"
Sep 07 12:08:38.010954 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:38.010889    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969" path="/var/lib/kubelet/pods/8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969/volumes"
Sep 07 12:08:38.011215 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:38.011187    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5eb1e62-13b1-4973-aa34-ff4839fea413" path="/var/lib/kubelet/pods/e5eb1e62-13b1-4973-aa34-ff4839fea413/volumes"
Sep 07 12:08:41.059607 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.059568    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp"]
Sep 07 12:08:41.059985 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.059904    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88619833-32c5-4418-8925-91d78f45518a" containerName="payload-pre-processing"
Sep 07 12:08:41.059985 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.059914    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="88619833-32c5-4418-8925-91d78f45518a" containerName="payload-pre-processing"
Sep 07 12:08:41.059985 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.059928    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5eb1e62-13b1-4973-aa34-ff4839fea413" containerName="payload-processing"
Sep 07 12:08:41.059985 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.059934    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5eb1e62-13b1-4973-aa34-ff4839fea413" containerName="payload-processing"
Sep 07 12:08:41.059985 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.059948    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969" containerName="revoke-keys"
Sep 07 12:08:41.059985 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.059954    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969" containerName="revoke-keys"
Sep 07 12:08:41.060234 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.060009    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="88619833-32c5-4418-8925-91d78f45518a" containerName="payload-pre-processing"
Sep 07 12:08:41.060234 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.060019    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c8d15e6-c7e1-43aa-bfe1-bd11b4d53969" containerName="revoke-keys"
Sep 07 12:08:41.060234 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.060027    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5eb1e62-13b1-4973-aa34-ff4839fea413" containerName="payload-processing"
Sep 07 12:08:41.064552 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.064530    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp"
Sep 07 12:08:41.066981 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.066957    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:08:41.067265 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.067245    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:08:41.072063 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.071895    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp"]
Sep 07 12:08:41.163017 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.162973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f17cb9fd-0236-4ac0-b324-4cc5156f67b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp\" (UID: \"f17cb9fd-0236-4ac0-b324-4cc5156f67b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp"
Sep 07 12:08:41.264890 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.264848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f17cb9fd-0236-4ac0-b324-4cc5156f67b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp\" (UID: \"f17cb9fd-0236-4ac0-b324-4cc5156f67b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp"
Sep 07 12:08:41.265523 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.265503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f17cb9fd-0236-4ac0-b324-4cc5156f67b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp\" (UID: \"f17cb9fd-0236-4ac0-b324-4cc5156f67b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp"
Sep 07 12:08:41.377384 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.377341    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp"
Sep 07 12:08:41.653971 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:41.653880    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp"]
Sep 07 12:08:41.657412 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:08:41.657383    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf17cb9fd_0236_4ac0_b324_4cc5156f67b9.slice/crio-8198952ff35774c09244d79842fa0174ac56d0b92e2d572f91c9baa4b852f238 WatchSource:0}: Error finding container 8198952ff35774c09244d79842fa0174ac56d0b92e2d572f91c9baa4b852f238: Status 404 returned error can't find the container with id 8198952ff35774c09244d79842fa0174ac56d0b92e2d572f91c9baa4b852f238
Sep 07 12:08:42.434225 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:42.434175    2580 generic.go:356] "Generic (PLEG): container finished" podID="f17cb9fd-0236-4ac0-b324-4cc5156f67b9" containerID="1ff464a6a6b750d1fab3d1a2ef12623ae4b3f80c493f6b62a3a33353f7cbd3cd" exitCode=0
Sep 07 12:08:42.434617 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:42.434263    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp" event={"ID":"f17cb9fd-0236-4ac0-b324-4cc5156f67b9","Type":"ContainerDied","Data":"1ff464a6a6b750d1fab3d1a2ef12623ae4b3f80c493f6b62a3a33353f7cbd3cd"}
Sep 07 12:08:42.434617 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:42.434349    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp" event={"ID":"f17cb9fd-0236-4ac0-b324-4cc5156f67b9","Type":"ContainerStarted","Data":"8198952ff35774c09244d79842fa0174ac56d0b92e2d572f91c9baa4b852f238"}
Sep 07 12:08:43.425611 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:43.425574    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"]
Sep 07 12:08:43.425829 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:43.425795    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62" podUID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" containerName="istio-proxy" containerID="cri-o://164c041e4e423c6813b0a3f3a7122d99229e9da7a0c64bb9882890b202e75bc5" gracePeriod=30
Sep 07 12:08:43.586602 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:43.586576    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp"
Sep 07 12:08:43.688812 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:43.688716    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f17cb9fd-0236-4ac0-b324-4cc5156f67b9-maas-api-service-ca\") pod \"f17cb9fd-0236-4ac0-b324-4cc5156f67b9\" (UID: \"f17cb9fd-0236-4ac0-b324-4cc5156f67b9\") "
Sep 07 12:08:43.689086 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:43.689063    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f17cb9fd-0236-4ac0-b324-4cc5156f67b9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f17cb9fd-0236-4ac0-b324-4cc5156f67b9" (UID: "f17cb9fd-0236-4ac0-b324-4cc5156f67b9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:08:43.759541 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:43.759497    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-af50c7-7h472"]
Sep 07 12:08:43.759866 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:43.759855    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f17cb9fd-0236-4ac0-b324-4cc5156f67b9" containerName="revoke-keys"
Sep 07 12:08:43.759915 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:43.759868    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f17cb9fd-0236-4ac0-b324-4cc5156f67b9" containerName="revoke-keys"
Sep 07 12:08:43.759950 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:43.759932    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f17cb9fd-0236-4ac0-b324-4cc5156f67b9" containerName="revoke-keys"
Sep 07 12:08:43.762902 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:43.762878    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-af50c7-7h472"
Sep 07 12:08:43.771232 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:43.771188    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-af50c7-7h472"]
Sep 07 12:08:43.789639 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:43.789609    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f17cb9fd-0236-4ac0-b324-4cc5156f67b9-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:43.890869 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:43.890825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f375ba01-40ad-48f2-9412-26e5d335a41d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-af50c7-7h472\" (UID: \"f375ba01-40ad-48f2-9412-26e5d335a41d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-af50c7-7h472"
Sep 07 12:08:43.992065 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:43.991957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f375ba01-40ad-48f2-9412-26e5d335a41d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-af50c7-7h472\" (UID: \"f375ba01-40ad-48f2-9412-26e5d335a41d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-af50c7-7h472"
Sep 07 12:08:43.992675 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:43.992652    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f375ba01-40ad-48f2-9412-26e5d335a41d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-af50c7-7h472\" (UID: \"f375ba01-40ad-48f2-9412-26e5d335a41d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-af50c7-7h472"
Sep 07 12:08:44.075503 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:44.075458    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-af50c7-7h472"
Sep 07 12:08:44.345969 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:44.345875    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-af50c7-7h472"]
Sep 07 12:08:44.348529 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:08:44.348491    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf375ba01_40ad_48f2_9412_26e5d335a41d.slice/crio-d67ecf228f84d6e4c1f1aa4a985a4c0023cec58a95739cc6d4f8e7efc11f9709 WatchSource:0}: Error finding container d67ecf228f84d6e4c1f1aa4a985a4c0023cec58a95739cc6d4f8e7efc11f9709: Status 404 returned error can't find the container with id d67ecf228f84d6e4c1f1aa4a985a4c0023cec58a95739cc6d4f8e7efc11f9709
Sep 07 12:08:44.447000 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:44.446969    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp"
Sep 07 12:08:44.447000 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:44.446975    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp" event={"ID":"f17cb9fd-0236-4ac0-b324-4cc5156f67b9","Type":"ContainerDied","Data":"8198952ff35774c09244d79842fa0174ac56d0b92e2d572f91c9baa4b852f238"}
Sep 07 12:08:44.447282 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:44.447025    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8198952ff35774c09244d79842fa0174ac56d0b92e2d572f91c9baa4b852f238"
Sep 07 12:08:44.448449 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:44.448418    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-af50c7-7h472" event={"ID":"f375ba01-40ad-48f2-9412-26e5d335a41d","Type":"ContainerStarted","Data":"d67ecf228f84d6e4c1f1aa4a985a4c0023cec58a95739cc6d4f8e7efc11f9709"}
Sep 07 12:08:45.454530 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:45.454496    2580 generic.go:356] "Generic (PLEG): container finished" podID="f375ba01-40ad-48f2-9412-26e5d335a41d" containerID="692ec19083f0c1c1bb94b60887432fd80d081244ad728ee4b29f8b6505d66a42" exitCode=0
Sep 07 12:08:45.454906 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:45.454543    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-af50c7-7h472" event={"ID":"f375ba01-40ad-48f2-9412-26e5d335a41d","Type":"ContainerDied","Data":"692ec19083f0c1c1bb94b60887432fd80d081244ad728ee4b29f8b6505d66a42"}
Sep 07 12:08:45.636411 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:45.636371    2580 patch_prober.go:28] interesting pod/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 12:08:45.636563 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:45.636430    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62" podUID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 12:08:46.592656 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:46.592631    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-af50c7-7h472"
Sep 07 12:08:46.615153 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:46.615107    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f375ba01-40ad-48f2-9412-26e5d335a41d-maas-api-service-ca\") pod \"f375ba01-40ad-48f2-9412-26e5d335a41d\" (UID: \"f375ba01-40ad-48f2-9412-26e5d335a41d\") "
Sep 07 12:08:46.615502 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:46.615481    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f375ba01-40ad-48f2-9412-26e5d335a41d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f375ba01-40ad-48f2-9412-26e5d335a41d" (UID: "f375ba01-40ad-48f2-9412-26e5d335a41d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:08:46.715920 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:46.715838    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f375ba01-40ad-48f2-9412-26e5d335a41d-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:47.466844 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:47.466813    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-af50c7-7h472"
Sep 07 12:08:47.466844 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:47.466830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-af50c7-7h472" event={"ID":"f375ba01-40ad-48f2-9412-26e5d335a41d","Type":"ContainerDied","Data":"d67ecf228f84d6e4c1f1aa4a985a4c0023cec58a95739cc6d4f8e7efc11f9709"}
Sep 07 12:08:47.467109 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:47.466884    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d67ecf228f84d6e4c1f1aa4a985a4c0023cec58a95739cc6d4f8e7efc11f9709"
Sep 07 12:08:48.675642 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.675607    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:08:48.733451 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.733355    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-token\") pod \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") "
Sep 07 12:08:48.733451 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.733401    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-envoy\") pod \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") "
Sep 07 12:08:48.733451 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.733434    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istiod-ca-cert\") pod \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") "
Sep 07 12:08:48.733732 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.733481    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-workload-certs\") pod \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") "
Sep 07 12:08:48.733732 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.733535    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-podinfo\") pod \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") "
Sep 07 12:08:48.733732 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.733581    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-data\") pod \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") "
Sep 07 12:08:48.733732 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.733614    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-credential-socket\") pod \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") "
Sep 07 12:08:48.733732 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.733651    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-workload-socket\") pod \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") "
Sep 07 12:08:48.733732 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.733697    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2skb5\" (UniqueName: \"kubernetes.io/projected/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-kube-api-access-2skb5\") pod \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\" (UID: \"6b47525c-a9fb-45a3-be5b-42b7e36f22eb\") "
Sep 07 12:08:48.734025 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.733833    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6b47525c-a9fb-45a3-be5b-42b7e36f22eb" (UID: "6b47525c-a9fb-45a3-be5b-42b7e36f22eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:48.734025 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.733884    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6b47525c-a9fb-45a3-be5b-42b7e36f22eb" (UID: "6b47525c-a9fb-45a3-be5b-42b7e36f22eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:08:48.734025 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.733888    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-data" (OuterVolumeSpecName: "istio-data") pod "6b47525c-a9fb-45a3-be5b-42b7e36f22eb" (UID: "6b47525c-a9fb-45a3-be5b-42b7e36f22eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:48.734025 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.734018    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6b47525c-a9fb-45a3-be5b-42b7e36f22eb" (UID: "6b47525c-a9fb-45a3-be5b-42b7e36f22eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:48.734291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.734034    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6b47525c-a9fb-45a3-be5b-42b7e36f22eb" (UID: "6b47525c-a9fb-45a3-be5b-42b7e36f22eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:48.734291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.734080    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istiod-ca-cert\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:48.734291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.734100    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-workload-certs\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:48.734291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.734115    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-data\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:48.735713 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.735678    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6b47525c-a9fb-45a3-be5b-42b7e36f22eb" (UID: "6b47525c-a9fb-45a3-be5b-42b7e36f22eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:08:48.735959 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.735939    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-kube-api-access-2skb5" (OuterVolumeSpecName: "kube-api-access-2skb5") pod "6b47525c-a9fb-45a3-be5b-42b7e36f22eb" (UID: "6b47525c-a9fb-45a3-be5b-42b7e36f22eb"). InnerVolumeSpecName "kube-api-access-2skb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:08:48.736064 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.736044    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6b47525c-a9fb-45a3-be5b-42b7e36f22eb" (UID: "6b47525c-a9fb-45a3-be5b-42b7e36f22eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:08:48.736125 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.736061    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-token" (OuterVolumeSpecName: "istio-token") pod "6b47525c-a9fb-45a3-be5b-42b7e36f22eb" (UID: "6b47525c-a9fb-45a3-be5b-42b7e36f22eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:08:48.835595 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.835551    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-credential-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:48.835595 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.835585    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-workload-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:48.835595 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.835594    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2skb5\" (UniqueName: \"kubernetes.io/projected/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-kube-api-access-2skb5\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:48.835595 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.835603    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-token\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:48.835859 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.835612    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-envoy\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:48.835859 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:48.835619    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b47525c-a9fb-45a3-be5b-42b7e36f22eb-istio-podinfo\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:49.478146 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:49.478107    2580 generic.go:356] "Generic (PLEG): container finished" podID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" containerID="164c041e4e423c6813b0a3f3a7122d99229e9da7a0c64bb9882890b202e75bc5" exitCode=0
Sep 07 12:08:49.478343 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:49.478155    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62" event={"ID":"6b47525c-a9fb-45a3-be5b-42b7e36f22eb","Type":"ContainerDied","Data":"164c041e4e423c6813b0a3f3a7122d99229e9da7a0c64bb9882890b202e75bc5"}
Sep 07 12:08:49.478343 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:49.478165    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"
Sep 07 12:08:49.478343 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:49.478231    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62" event={"ID":"6b47525c-a9fb-45a3-be5b-42b7e36f22eb","Type":"ContainerDied","Data":"c8c694a95bf1345be1860d68e6cd25e0d145649ee1a9e594183b8cc1a97def15"}
Sep 07 12:08:49.478343 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:49.478266    2580 scope.go:117] "RemoveContainer" containerID="164c041e4e423c6813b0a3f3a7122d99229e9da7a0c64bb9882890b202e75bc5"
Sep 07 12:08:49.487485 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:49.487464    2580 scope.go:117] "RemoveContainer" containerID="6cfd1e4a4ac74b944d6417420abb3eca3275962ba1f814483e3eefb03d672403"
Sep 07 12:08:49.495684 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:49.495664    2580 scope.go:117] "RemoveContainer" containerID="164c041e4e423c6813b0a3f3a7122d99229e9da7a0c64bb9882890b202e75bc5"
Sep 07 12:08:49.496021 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:08:49.496000    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"164c041e4e423c6813b0a3f3a7122d99229e9da7a0c64bb9882890b202e75bc5\": container with ID starting with 164c041e4e423c6813b0a3f3a7122d99229e9da7a0c64bb9882890b202e75bc5 not found: ID does not exist" containerID="164c041e4e423c6813b0a3f3a7122d99229e9da7a0c64bb9882890b202e75bc5"
Sep 07 12:08:49.496081 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:49.496030    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"164c041e4e423c6813b0a3f3a7122d99229e9da7a0c64bb9882890b202e75bc5"} err="failed to get container status \"164c041e4e423c6813b0a3f3a7122d99229e9da7a0c64bb9882890b202e75bc5\": rpc error: code = NotFound desc = could not find container \"164c041e4e423c6813b0a3f3a7122d99229e9da7a0c64bb9882890b202e75bc5\": container with ID starting with 164c041e4e423c6813b0a3f3a7122d99229e9da7a0c64bb9882890b202e75bc5 not found: ID does not exist"
Sep 07 12:08:49.496081 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:49.496052    2580 scope.go:117] "RemoveContainer" containerID="6cfd1e4a4ac74b944d6417420abb3eca3275962ba1f814483e3eefb03d672403"
Sep 07 12:08:49.496324 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:08:49.496305    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cfd1e4a4ac74b944d6417420abb3eca3275962ba1f814483e3eefb03d672403\": container with ID starting with 6cfd1e4a4ac74b944d6417420abb3eca3275962ba1f814483e3eefb03d672403 not found: ID does not exist" containerID="6cfd1e4a4ac74b944d6417420abb3eca3275962ba1f814483e3eefb03d672403"
Sep 07 12:08:49.496379 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:49.496333    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cfd1e4a4ac74b944d6417420abb3eca3275962ba1f814483e3eefb03d672403"} err="failed to get container status \"6cfd1e4a4ac74b944d6417420abb3eca3275962ba1f814483e3eefb03d672403\": rpc error: code = NotFound desc = could not find container \"6cfd1e4a4ac74b944d6417420abb3eca3275962ba1f814483e3eefb03d672403\": container with ID starting with 6cfd1e4a4ac74b944d6417420abb3eca3275962ba1f814483e3eefb03d672403 not found: ID does not exist"
Sep 07 12:08:49.500029 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:49.500002    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"]
Sep 07 12:08:49.503307 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:49.503284    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c7d751-openshift-default-59dbf97456-7mq62"]
Sep 07 12:08:50.009572 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:50.009536    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" path="/var/lib/kubelet/pods/6b47525c-a9fb-45a3-be5b-42b7e36f22eb/volumes"
Sep 07 12:08:51.814422 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:51.814386    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"]
Sep 07 12:08:51.814897 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:51.814602    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24" podUID="b133963b-1833-4846-a668-f50a6f124cae" containerName="payload-processing" containerID="cri-o://8e8023624cd775da5680af0dd63dc708398494405dcd7ed9542f1a0b0453794a" gracePeriod=30
Sep 07 12:08:51.990246 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:51.990193    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp"]
Sep 07 12:08:51.992727 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:51.992703    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f62f6f8-9zcgp"]
Sep 07 12:08:52.011109 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.011075    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f17cb9fd-0236-4ac0-b324-4cc5156f67b9" path="/var/lib/kubelet/pods/f17cb9fd-0236-4ac0-b324-4cc5156f67b9/volumes"
Sep 07 12:08:52.068009 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.067932    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"
Sep 07 12:08:52.165133 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.165097    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbz4n\" (UniqueName: \"kubernetes.io/projected/b133963b-1833-4846-a668-f50a6f124cae-kube-api-access-sbz4n\") pod \"b133963b-1833-4846-a668-f50a6f124cae\" (UID: \"b133963b-1833-4846-a668-f50a6f124cae\") "
Sep 07 12:08:52.165133 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.165136    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b133963b-1833-4846-a668-f50a6f124cae-plugins-config-volume\") pod \"b133963b-1833-4846-a668-f50a6f124cae\" (UID: \"b133963b-1833-4846-a668-f50a6f124cae\") "
Sep 07 12:08:52.165533 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.165513    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b133963b-1833-4846-a668-f50a6f124cae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b133963b-1833-4846-a668-f50a6f124cae" (UID: "b133963b-1833-4846-a668-f50a6f124cae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:08:52.167254 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.167229    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b133963b-1833-4846-a668-f50a6f124cae-kube-api-access-sbz4n" (OuterVolumeSpecName: "kube-api-access-sbz4n") pod "b133963b-1833-4846-a668-f50a6f124cae" (UID: "b133963b-1833-4846-a668-f50a6f124cae"). InnerVolumeSpecName "kube-api-access-sbz4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:08:52.266315 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.266277    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbz4n\" (UniqueName: \"kubernetes.io/projected/b133963b-1833-4846-a668-f50a6f124cae-kube-api-access-sbz4n\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:52.266315 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.266309    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b133963b-1833-4846-a668-f50a6f124cae-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:52.494776 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.494745    2580 generic.go:356] "Generic (PLEG): container finished" podID="b133963b-1833-4846-a668-f50a6f124cae" containerID="8e8023624cd775da5680af0dd63dc708398494405dcd7ed9542f1a0b0453794a" exitCode=0
Sep 07 12:08:52.494974 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.494787    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"
Sep 07 12:08:52.494974 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.494835    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24" event={"ID":"b133963b-1833-4846-a668-f50a6f124cae","Type":"ContainerDied","Data":"8e8023624cd775da5680af0dd63dc708398494405dcd7ed9542f1a0b0453794a"}
Sep 07 12:08:52.494974 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.494885    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24" event={"ID":"b133963b-1833-4846-a668-f50a6f124cae","Type":"ContainerDied","Data":"f2d5edd12ca191df5516b80fe8ba66bcb6ff3f9b356d7b35dba62935869239cf"}
Sep 07 12:08:52.494974 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.494910    2580 scope.go:117] "RemoveContainer" containerID="8e8023624cd775da5680af0dd63dc708398494405dcd7ed9542f1a0b0453794a"
Sep 07 12:08:52.504350 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.504329    2580 scope.go:117] "RemoveContainer" containerID="8e8023624cd775da5680af0dd63dc708398494405dcd7ed9542f1a0b0453794a"
Sep 07 12:08:52.504684 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:08:52.504664    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e8023624cd775da5680af0dd63dc708398494405dcd7ed9542f1a0b0453794a\": container with ID starting with 8e8023624cd775da5680af0dd63dc708398494405dcd7ed9542f1a0b0453794a not found: ID does not exist" containerID="8e8023624cd775da5680af0dd63dc708398494405dcd7ed9542f1a0b0453794a"
Sep 07 12:08:52.504721 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.504692    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e8023624cd775da5680af0dd63dc708398494405dcd7ed9542f1a0b0453794a"} err="failed to get container status \"8e8023624cd775da5680af0dd63dc708398494405dcd7ed9542f1a0b0453794a\": rpc error: code = NotFound desc = could not find container \"8e8023624cd775da5680af0dd63dc708398494405dcd7ed9542f1a0b0453794a\": container with ID starting with 8e8023624cd775da5680af0dd63dc708398494405dcd7ed9542f1a0b0453794a not found: ID does not exist"
Sep 07 12:08:52.518155 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.517389    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"]
Sep 07 12:08:52.523477 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:52.523447    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5f62f6f8-9d6bf98f7-f8z24"]
Sep 07 12:08:54.011540 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:54.011497    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b133963b-1833-4846-a668-f50a6f124cae" path="/var/lib/kubelet/pods/b133963b-1833-4846-a668-f50a6f124cae/volumes"
Sep 07 12:08:54.634434 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:54.634388    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"]
Sep 07 12:08:54.634717 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:54.634689    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj" podUID="34536cde-c715-4276-bf49-8a25d8f21f7f" containerName="payload-processing" containerID="cri-o://da45089362dac8080fbc0bf6778f4b7eb55de24d186b5588d042e15b394c6f5b" gracePeriod=30
Sep 07 12:08:54.802527 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:54.802493    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-af50c7-7h472"]
Sep 07 12:08:54.804877 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:54.804852    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-af50c7-7h472"]
Sep 07 12:08:54.909136 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:54.909109    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"
Sep 07 12:08:54.991924 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:54.991828    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctqbk\" (UniqueName: \"kubernetes.io/projected/34536cde-c715-4276-bf49-8a25d8f21f7f-kube-api-access-ctqbk\") pod \"34536cde-c715-4276-bf49-8a25d8f21f7f\" (UID: \"34536cde-c715-4276-bf49-8a25d8f21f7f\") "
Sep 07 12:08:54.992123 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:54.992064    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34536cde-c715-4276-bf49-8a25d8f21f7f-plugins-config-volume\") pod \"34536cde-c715-4276-bf49-8a25d8f21f7f\" (UID: \"34536cde-c715-4276-bf49-8a25d8f21f7f\") "
Sep 07 12:08:54.992424 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:54.992399    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34536cde-c715-4276-bf49-8a25d8f21f7f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "34536cde-c715-4276-bf49-8a25d8f21f7f" (UID: "34536cde-c715-4276-bf49-8a25d8f21f7f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:08:54.993921 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:54.993892    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34536cde-c715-4276-bf49-8a25d8f21f7f-kube-api-access-ctqbk" (OuterVolumeSpecName: "kube-api-access-ctqbk") pod "34536cde-c715-4276-bf49-8a25d8f21f7f" (UID: "34536cde-c715-4276-bf49-8a25d8f21f7f"). InnerVolumeSpecName "kube-api-access-ctqbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:08:55.093293 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:55.093255    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34536cde-c715-4276-bf49-8a25d8f21f7f-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:55.093293 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:55.093285    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctqbk\" (UniqueName: \"kubernetes.io/projected/34536cde-c715-4276-bf49-8a25d8f21f7f-kube-api-access-ctqbk\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:08:55.513038 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:55.513006    2580 generic.go:356] "Generic (PLEG): container finished" podID="34536cde-c715-4276-bf49-8a25d8f21f7f" containerID="da45089362dac8080fbc0bf6778f4b7eb55de24d186b5588d042e15b394c6f5b" exitCode=0
Sep 07 12:08:55.513186 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:55.513054    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"
Sep 07 12:08:55.513186 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:55.513097    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj" event={"ID":"34536cde-c715-4276-bf49-8a25d8f21f7f","Type":"ContainerDied","Data":"da45089362dac8080fbc0bf6778f4b7eb55de24d186b5588d042e15b394c6f5b"}
Sep 07 12:08:55.513186 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:55.513155    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj" event={"ID":"34536cde-c715-4276-bf49-8a25d8f21f7f","Type":"ContainerDied","Data":"dd3b5556370112541d16207fb31677272653a262e5a12a02a6571e67e3511b8d"}
Sep 07 12:08:55.513186 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:55.513183    2580 scope.go:117] "RemoveContainer" containerID="da45089362dac8080fbc0bf6778f4b7eb55de24d186b5588d042e15b394c6f5b"
Sep 07 12:08:55.522396 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:55.522380    2580 scope.go:117] "RemoveContainer" containerID="da45089362dac8080fbc0bf6778f4b7eb55de24d186b5588d042e15b394c6f5b"
Sep 07 12:08:55.522697 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:08:55.522675    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da45089362dac8080fbc0bf6778f4b7eb55de24d186b5588d042e15b394c6f5b\": container with ID starting with da45089362dac8080fbc0bf6778f4b7eb55de24d186b5588d042e15b394c6f5b not found: ID does not exist" containerID="da45089362dac8080fbc0bf6778f4b7eb55de24d186b5588d042e15b394c6f5b"
Sep 07 12:08:55.522747 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:55.522706    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da45089362dac8080fbc0bf6778f4b7eb55de24d186b5588d042e15b394c6f5b"} err="failed to get container status \"da45089362dac8080fbc0bf6778f4b7eb55de24d186b5588d042e15b394c6f5b\": rpc error: code = NotFound desc = could not find container \"da45089362dac8080fbc0bf6778f4b7eb55de24d186b5588d042e15b394c6f5b\": container with ID starting with da45089362dac8080fbc0bf6778f4b7eb55de24d186b5588d042e15b394c6f5b not found: ID does not exist"
Sep 07 12:08:55.533703 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:55.533663    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"]
Sep 07 12:08:55.539246 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:55.539218    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-af50c7-66d8b6c967-6t7fj"]
Sep 07 12:08:56.009321 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:56.009286    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34536cde-c715-4276-bf49-8a25d8f21f7f" path="/var/lib/kubelet/pods/34536cde-c715-4276-bf49-8a25d8f21f7f/volumes"
Sep 07 12:08:56.009621 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:56.009608    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f375ba01-40ad-48f2-9412-26e5d335a41d" path="/var/lib/kubelet/pods/f375ba01-40ad-48f2-9412-26e5d335a41d/volumes"
Sep 07 12:08:58.460679 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:58.460645    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"]
Sep 07 12:08:58.461118 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:08:58.460871    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks" podUID="5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" containerName="istio-proxy" containerID="cri-o://64edd28da7c5dfe4add69b0b129ff828837db6fb6cc71f0885fba8b4eb87e296" gracePeriod=30
Sep 07 12:09:01.876114 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.876082    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"]
Sep 07 12:09:01.876610 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.876558    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" containerName="istio-proxy"
Sep 07 12:09:01.876610 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.876578    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" containerName="istio-proxy"
Sep 07 12:09:01.876610 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.876589    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b133963b-1833-4846-a668-f50a6f124cae" containerName="payload-processing"
Sep 07 12:09:01.876610 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.876596    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b133963b-1833-4846-a668-f50a6f124cae" containerName="payload-processing"
Sep 07 12:09:01.876610 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.876608    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34536cde-c715-4276-bf49-8a25d8f21f7f" containerName="payload-processing"
Sep 07 12:09:01.876859 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.876617    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="34536cde-c715-4276-bf49-8a25d8f21f7f" containerName="payload-processing"
Sep 07 12:09:01.876859 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.876628    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f375ba01-40ad-48f2-9412-26e5d335a41d" containerName="revoke-keys"
Sep 07 12:09:01.876859 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.876637    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f375ba01-40ad-48f2-9412-26e5d335a41d" containerName="revoke-keys"
Sep 07 12:09:01.876859 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.876732    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="34536cde-c715-4276-bf49-8a25d8f21f7f" containerName="payload-processing"
Sep 07 12:09:01.876859 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.876744    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b133963b-1833-4846-a668-f50a6f124cae" containerName="payload-processing"
Sep 07 12:09:01.876859 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.876755    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f375ba01-40ad-48f2-9412-26e5d335a41d" containerName="revoke-keys"
Sep 07 12:09:01.876859 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.876764    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" containerName="istio-proxy"
Sep 07 12:09:01.881369 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.881348    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:01.883856 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.883834    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-491296-openshift-default-dockercfg-6hbs6\""
Sep 07 12:09:01.890683 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.890651    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"]
Sep 07 12:09:01.952581 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.952543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istiod-ca-cert\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:01.952581 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.952584    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-envoy\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:01.952788 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.952610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-podinfo\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:01.952788 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.952637    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-workload-certs\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:01.952788 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.952726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-workload-socket\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:01.952788 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.952755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-token\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:01.952788 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.952781    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-data\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:01.952965 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.952881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qzfl\" (UniqueName: \"kubernetes.io/projected/d52eecbf-1cd0-44ec-822b-b31cc579adbb-kube-api-access-5qzfl\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:01.952965 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:01.952922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-credential-socket\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.053417 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.053372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qzfl\" (UniqueName: \"kubernetes.io/projected/d52eecbf-1cd0-44ec-822b-b31cc579adbb-kube-api-access-5qzfl\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.053577 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.053443    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-credential-socket\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.053577 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.053463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istiod-ca-cert\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.053577 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.053476    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-envoy\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.053577 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.053505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-podinfo\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.053577 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.053527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-workload-certs\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.053829 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.053582    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-workload-socket\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.053829 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.053607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-token\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.053829 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.053633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-data\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.054154 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.054123    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-workload-certs\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.054399 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.054370    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-credential-socket\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.054509 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.054445    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-workload-socket\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.054509 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.054459    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-data\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.054509 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.054481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istiod-ca-cert\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.056131 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.056103    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-envoy\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.056458 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.056436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-podinfo\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.062309 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.062282    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-token\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.062423 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.062358    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qzfl\" (UniqueName: \"kubernetes.io/projected/d52eecbf-1cd0-44ec-822b-b31cc579adbb-kube-api-access-5qzfl\") pod \"e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") " pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.192933 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.192836    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:02.479072 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.479043    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"]
Sep 07 12:09:02.481741 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:09:02.481706    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd52eecbf_1cd0_44ec_822b_b31cc579adbb.slice/crio-c619b248f196d8002ba4e1a42934ea571b2a05d864cfa5bac0dc3bff176fec0a WatchSource:0}: Error finding container c619b248f196d8002ba4e1a42934ea571b2a05d864cfa5bac0dc3bff176fec0a: Status 404 returned error can't find the container with id c619b248f196d8002ba4e1a42934ea571b2a05d864cfa5bac0dc3bff176fec0a
Sep 07 12:09:02.484065 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.484028    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:02.484128 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.484113    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:02.484171 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.484156    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:02.558276 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:02.558236    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4" event={"ID":"d52eecbf-1cd0-44ec-822b-b31cc579adbb","Type":"ContainerStarted","Data":"c619b248f196d8002ba4e1a42934ea571b2a05d864cfa5bac0dc3bff176fec0a"}
Sep 07 12:09:03.564728 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.564694    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4" event={"ID":"d52eecbf-1cd0-44ec-822b-b31cc579adbb","Type":"ContainerStarted","Data":"279420f47cf89a444c36578368328f5836714fb2f51ecda771b4891727f32e00"}
Sep 07 12:09:03.585164 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.585102    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4" podStartSLOduration=2.585078927 podStartE2EDuration="2.585078927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:09:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:09:03.583253667 +0000 UTC m=+1298.079143091" watchObservedRunningTime="2026-09-07 12:09:03.585078927 +0000 UTC m=+1298.080968348"
Sep 07 12:09:03.712716 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.712688    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:09:03.768803 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.768703    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9r6kx\" (UniqueName: \"kubernetes.io/projected/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-kube-api-access-9r6kx\") pod \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") "
Sep 07 12:09:03.768803 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.768759    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-workload-certs\") pod \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") "
Sep 07 12:09:03.768803 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.768794    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-credential-socket\") pod \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") "
Sep 07 12:09:03.769084 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.768824    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istiod-ca-cert\") pod \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") "
Sep 07 12:09:03.769084 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.768854    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-podinfo\") pod \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") "
Sep 07 12:09:03.769084 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.768899    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-envoy\") pod \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") "
Sep 07 12:09:03.769084 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.768956    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-token\") pod \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") "
Sep 07 12:09:03.769084 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.769010    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-workload-socket\") pod \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") "
Sep 07 12:09:03.769084 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.769063    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-data\") pod \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\" (UID: \"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb\") "
Sep 07 12:09:03.769402 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.769133    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" (UID: "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:09:03.769402 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.769137    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" (UID: "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:09:03.769402 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.769373    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" (UID: "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:09:03.769556 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.769396    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" (UID: "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:09:03.769556 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.769404    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-data" (OuterVolumeSpecName: "istio-data") pod "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" (UID: "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:09:03.769556 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.769480    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-workload-certs\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:09:03.769556 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.769499    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-credential-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:09:03.769556 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.769514    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-workload-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:09:03.771066 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.771037    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-kube-api-access-9r6kx" (OuterVolumeSpecName: "kube-api-access-9r6kx") pod "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" (UID: "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb"). InnerVolumeSpecName "kube-api-access-9r6kx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:09:03.771165 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.771106    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-token" (OuterVolumeSpecName: "istio-token") pod "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" (UID: "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:09:03.771464 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.771436    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" (UID: "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:09:03.771464 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.771455    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" (UID: "5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:09:03.870662 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.870619    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r6kx\" (UniqueName: \"kubernetes.io/projected/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-kube-api-access-9r6kx\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:09:03.870662 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.870652    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istiod-ca-cert\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:09:03.870662 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.870664    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-podinfo\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:09:03.870662 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.870672    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-envoy\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:09:03.870930 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.870679    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-token\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:09:03.870930 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:03.870687    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb-istio-data\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:09:04.193908 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:04.193870    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:04.198811 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:04.198789    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:04.571267 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:04.571154    2580 generic.go:356] "Generic (PLEG): container finished" podID="5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" containerID="64edd28da7c5dfe4add69b0b129ff828837db6fb6cc71f0885fba8b4eb87e296" exitCode=0
Sep 07 12:09:04.571267 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:04.571231    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"
Sep 07 12:09:04.571267 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:04.571242    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks" event={"ID":"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb","Type":"ContainerDied","Data":"64edd28da7c5dfe4add69b0b129ff828837db6fb6cc71f0885fba8b4eb87e296"}
Sep 07 12:09:04.571758 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:04.571305    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks" event={"ID":"5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb","Type":"ContainerDied","Data":"ee54c925339a8911973b55f929ec035f51dc7f916c5033a0fe6396f59599e18e"}
Sep 07 12:09:04.571758 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:04.571334    2580 scope.go:117] "RemoveContainer" containerID="64edd28da7c5dfe4add69b0b129ff828837db6fb6cc71f0885fba8b4eb87e296"
Sep 07 12:09:04.571758 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:04.571563    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:04.572755 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:04.572738    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:04.580799 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:04.580635    2580 scope.go:117] "RemoveContainer" containerID="48051f2ec6e722adb4f95db13bdf9fb1ee140b85b31b65a92519ed86df26c1c6"
Sep 07 12:09:04.587501 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:04.587472    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"]
Sep 07 12:09:04.589591 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:04.589566    2580 scope.go:117] "RemoveContainer" containerID="64edd28da7c5dfe4add69b0b129ff828837db6fb6cc71f0885fba8b4eb87e296"
Sep 07 12:09:04.589958 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:09:04.589939    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64edd28da7c5dfe4add69b0b129ff828837db6fb6cc71f0885fba8b4eb87e296\": container with ID starting with 64edd28da7c5dfe4add69b0b129ff828837db6fb6cc71f0885fba8b4eb87e296 not found: ID does not exist" containerID="64edd28da7c5dfe4add69b0b129ff828837db6fb6cc71f0885fba8b4eb87e296"
Sep 07 12:09:04.590031 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:04.589965    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64edd28da7c5dfe4add69b0b129ff828837db6fb6cc71f0885fba8b4eb87e296"} err="failed to get container status \"64edd28da7c5dfe4add69b0b129ff828837db6fb6cc71f0885fba8b4eb87e296\": rpc error: code = NotFound desc = could not find container \"64edd28da7c5dfe4add69b0b129ff828837db6fb6cc71f0885fba8b4eb87e296\": container with ID starting with 64edd28da7c5dfe4add69b0b129ff828837db6fb6cc71f0885fba8b4eb87e296 not found: ID does not exist"
Sep 07 12:09:04.590031 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:04.589982    2580 scope.go:117] "RemoveContainer" containerID="48051f2ec6e722adb4f95db13bdf9fb1ee140b85b31b65a92519ed86df26c1c6"
Sep 07 12:09:04.590236 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:09:04.590219    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48051f2ec6e722adb4f95db13bdf9fb1ee140b85b31b65a92519ed86df26c1c6\": container with ID starting with 48051f2ec6e722adb4f95db13bdf9fb1ee140b85b31b65a92519ed86df26c1c6 not found: ID does not exist" containerID="48051f2ec6e722adb4f95db13bdf9fb1ee140b85b31b65a92519ed86df26c1c6"
Sep 07 12:09:04.590282 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:04.590241    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48051f2ec6e722adb4f95db13bdf9fb1ee140b85b31b65a92519ed86df26c1c6"} err="failed to get container status \"48051f2ec6e722adb4f95db13bdf9fb1ee140b85b31b65a92519ed86df26c1c6\": rpc error: code = NotFound desc = could not find container \"48051f2ec6e722adb4f95db13bdf9fb1ee140b85b31b65a92519ed86df26c1c6\": container with ID starting with 48051f2ec6e722adb4f95db13bdf9fb1ee140b85b31b65a92519ed86df26c1c6 not found: ID does not exist"
Sep 07 12:09:04.590942 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:04.590924    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5f62f6f8-openshift-default-59857c8784-d7hks"]
Sep 07 12:09:06.011366 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:06.011331    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" path="/var/lib/kubelet/pods/5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb/volumes"
Sep 07 12:09:07.252266 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.252234    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"]
Sep 07 12:09:07.252669 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.252539    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" containerName="istio-proxy"
Sep 07 12:09:07.252669 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.252550    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" containerName="istio-proxy"
Sep 07 12:09:07.252669 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.252561    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" containerName="istio-proxy"
Sep 07 12:09:07.252669 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.252568    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" containerName="istio-proxy"
Sep 07 12:09:07.252669 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.252627    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" containerName="istio-proxy"
Sep 07 12:09:07.252669 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.252638    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" containerName="istio-proxy"
Sep 07 12:09:07.252669 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.252645    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b47525c-a9fb-45a3-be5b-42b7e36f22eb" containerName="istio-proxy"
Sep 07 12:09:07.256600 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.256578    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"
Sep 07 12:09:07.259008 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.258975    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-491296\""
Sep 07 12:09:07.259119 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.259024    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-491296-dockercfg-xb9jl\""
Sep 07 12:09:07.266142 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.266115    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"]
Sep 07 12:09:07.303416 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.303375    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2b2c6f4-80ca-435f-98fe-4eca94dca98a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh\" (UID: \"e2b2c6f4-80ca-435f-98fe-4eca94dca98a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"
Sep 07 12:09:07.303416 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.303417    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shqkz\" (UniqueName: \"kubernetes.io/projected/e2b2c6f4-80ca-435f-98fe-4eca94dca98a-kube-api-access-shqkz\") pod \"payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh\" (UID: \"e2b2c6f4-80ca-435f-98fe-4eca94dca98a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"
Sep 07 12:09:07.404880 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.404831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2b2c6f4-80ca-435f-98fe-4eca94dca98a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh\" (UID: \"e2b2c6f4-80ca-435f-98fe-4eca94dca98a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"
Sep 07 12:09:07.404880 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.404880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shqkz\" (UniqueName: \"kubernetes.io/projected/e2b2c6f4-80ca-435f-98fe-4eca94dca98a-kube-api-access-shqkz\") pod \"payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh\" (UID: \"e2b2c6f4-80ca-435f-98fe-4eca94dca98a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"
Sep 07 12:09:07.410627 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.406045    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2b2c6f4-80ca-435f-98fe-4eca94dca98a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh\" (UID: \"e2b2c6f4-80ca-435f-98fe-4eca94dca98a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"
Sep 07 12:09:07.413772 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.413744    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shqkz\" (UniqueName: \"kubernetes.io/projected/e2b2c6f4-80ca-435f-98fe-4eca94dca98a-kube-api-access-shqkz\") pod \"payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh\" (UID: \"e2b2c6f4-80ca-435f-98fe-4eca94dca98a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"
Sep 07 12:09:07.568013 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.567912    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"
Sep 07 12:09:07.838013 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:07.837981    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"]
Sep 07 12:09:07.840802 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:09:07.840772    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2b2c6f4_80ca_435f_98fe_4eca94dca98a.slice/crio-595038f2343e529de26199f40e128b6b6a8341adb8a9fe2a54704acc8aee88fd WatchSource:0}: Error finding container 595038f2343e529de26199f40e128b6b6a8341adb8a9fe2a54704acc8aee88fd: Status 404 returned error can't find the container with id 595038f2343e529de26199f40e128b6b6a8341adb8a9fe2a54704acc8aee88fd
Sep 07 12:09:08.594642 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:08.594610    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh" event={"ID":"e2b2c6f4-80ca-435f-98fe-4eca94dca98a","Type":"ContainerStarted","Data":"d526ea8152c5a37f345f03ecc1972affa27bf932979904ee76d69bdbee90e19a"}
Sep 07 12:09:08.595166 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:08.594662    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh" event={"ID":"e2b2c6f4-80ca-435f-98fe-4eca94dca98a","Type":"ContainerStarted","Data":"595038f2343e529de26199f40e128b6b6a8341adb8a9fe2a54704acc8aee88fd"}
Sep 07 12:09:08.595166 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:08.594742    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"
Sep 07 12:09:08.610241 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:08.610156    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh" podStartSLOduration=1.610135825 podStartE2EDuration="1.610135825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:09:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:09:08.609361346 +0000 UTC m=+1303.105250770" watchObservedRunningTime="2026-09-07 12:09:08.610135825 +0000 UTC m=+1303.106025248"
Sep 07 12:09:12.181524 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.181486    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"]
Sep 07 12:09:12.181950 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.181935    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" containerName="istio-proxy"
Sep 07 12:09:12.182009 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.181951    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b5bb7df-eff7-4a64-9c3a-89e2cd38fadb" containerName="istio-proxy"
Sep 07 12:09:12.185023 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.185003    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.187511 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.187490    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-3a0bb2-openshift-default-dockercfg-7f2j8\""
Sep 07 12:09:12.195025 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.195001    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"]
Sep 07 12:09:12.251597 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.251564    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-podinfo\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.251597 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.251604    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj9kf\" (UniqueName: \"kubernetes.io/projected/3077305f-0902-4007-a3e6-beabfbb2a12d-kube-api-access-pj9kf\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.251852 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.251646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3077305f-0902-4007-a3e6-beabfbb2a12d-istiod-ca-cert\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.251852 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.251692    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-token\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.251852 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.251731    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-data\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.251852 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.251779    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-envoy\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.251852 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.251813    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-workload-certs\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.252082 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.251872    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-workload-socket\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.252082 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.251930    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-credential-socket\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.352808 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.352763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-credential-socket\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.352808 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.352814    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-podinfo\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.353056 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.352905    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj9kf\" (UniqueName: \"kubernetes.io/projected/3077305f-0902-4007-a3e6-beabfbb2a12d-kube-api-access-pj9kf\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.353056 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.352938    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3077305f-0902-4007-a3e6-beabfbb2a12d-istiod-ca-cert\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.353056 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.352972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-token\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.353056 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.353008    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-data\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.353306 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.353058    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-envoy\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.353306 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.353091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-workload-certs\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.353306 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.353123    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-workload-socket\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.353306 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.353283    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-credential-socket\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.353518 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.353436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-workload-socket\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.353577 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.353557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-data\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.353792 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.353758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3077305f-0902-4007-a3e6-beabfbb2a12d-istiod-ca-cert\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.353792 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.353758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-workload-certs\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.355641 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.355619    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-podinfo\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.355802 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.355783    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-envoy\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.361610 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.361580    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-token\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.363414 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.363393    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj9kf\" (UniqueName: \"kubernetes.io/projected/3077305f-0902-4007-a3e6-beabfbb2a12d-kube-api-access-pj9kf\") pod \"e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") " pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.497658 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.497450    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:12.810984 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.810958    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"]
Sep 07 12:09:12.812935 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:09:12.812904    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3077305f_0902_4007_a3e6_beabfbb2a12d.slice/crio-e7b00817fedc3d90440da57d4c465bb5b910de8f4bbd9775af4c563292ed588e WatchSource:0}: Error finding container e7b00817fedc3d90440da57d4c465bb5b910de8f4bbd9775af4c563292ed588e: Status 404 returned error can't find the container with id e7b00817fedc3d90440da57d4c465bb5b910de8f4bbd9775af4c563292ed588e
Sep 07 12:09:12.815250 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.815190    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:12.815336 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.815281    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:12.815336 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:12.815311    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:13.620736 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:13.620700    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf" event={"ID":"3077305f-0902-4007-a3e6-beabfbb2a12d","Type":"ContainerStarted","Data":"79050e3f9288b209747c68d170ef93552d85d58f3374f577d5c22f546dfb5bf6"}
Sep 07 12:09:13.621236 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:13.620788    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf" event={"ID":"3077305f-0902-4007-a3e6-beabfbb2a12d","Type":"ContainerStarted","Data":"e7b00817fedc3d90440da57d4c465bb5b910de8f4bbd9775af4c563292ed588e"}
Sep 07 12:09:13.640064 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:13.639996    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf" podStartSLOduration=1.6399752699999999 podStartE2EDuration="1.63997527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:09:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:09:13.639164851 +0000 UTC m=+1308.135054275" watchObservedRunningTime="2026-09-07 12:09:13.63997527 +0000 UTC m=+1308.135864693"
Sep 07 12:09:14.498093 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:14.498053    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:14.502666 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:14.502639    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:14.626418 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:14.626388    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:14.627411 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:14.627392    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:19.602186 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:19.602151    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"
Sep 07 12:09:20.613432 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:20.613385    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 07 12:09:20.613833 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:20.613466    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4" podUID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 07 12:09:21.664430 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:21.664399    2580 generic.go:356] "Generic (PLEG): container finished" podID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" containerID="279420f47cf89a444c36578368328f5836714fb2f51ecda771b4891727f32e00" exitCode=0
Sep 07 12:09:21.664854 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:21.664474    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4" event={"ID":"d52eecbf-1cd0-44ec-822b-b31cc579adbb","Type":"ContainerDied","Data":"279420f47cf89a444c36578368328f5836714fb2f51ecda771b4891727f32e00"}
Sep 07 12:09:21.664903 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:21.664868    2580 scope.go:117] "RemoveContainer" containerID="279420f47cf89a444c36578368328f5836714fb2f51ecda771b4891727f32e00"
Sep 07 12:09:21.665918 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:21.665892    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:21.666017 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:21.665976    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:21.666136 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:21.666025    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:22.678346 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:22.677472    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4" event={"ID":"d52eecbf-1cd0-44ec-822b-b31cc579adbb","Type":"ContainerStarted","Data":"d1b09100806082c28492da273520c3e74cf0931e5a32358bd8ee74c2b33772ae"}
Sep 07 12:09:23.193588 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:23.193558    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:23.193774 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:23.193718    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 07 12:09:23.193774 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:23.193763    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4" podUID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 07 12:09:24.193512 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:24.193467    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 07 12:09:24.194137 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:24.194100    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4" podUID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 07 12:09:25.194110 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:25.194071    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 07 12:09:25.194782 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:25.194744    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4" podUID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 07 12:09:26.193946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:26.193899    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 07 12:09:26.194151 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:26.193986    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4" podUID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 07 12:09:27.194104 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:27.194064    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 07 12:09:27.194863 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:27.194412    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4" podUID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 07 12:09:28.482583 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:28.482546    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:28.483082 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:28.482940    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:28.483888 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:28.483867    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:09:30.668383 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:30.668342    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 07 12:09:30.668773 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:30.668420    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf" podUID="3077305f-0902-4007-a3e6-beabfbb2a12d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 07 12:09:30.948189 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:30.948101    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"]
Sep 07 12:09:30.952517 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:30.952490    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:30.955006 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:30.954986    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-613886-openshift-default-dockercfg-rkh7c\""
Sep 07 12:09:30.965343 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:30.965318    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"]
Sep 07 12:09:31.051706 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.051672    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g2s6\" (UniqueName: \"kubernetes.io/projected/554cfa21-b6f6-44e2-9d05-c46047bb3539-kube-api-access-6g2s6\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.051706 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.051709    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-data\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.051966 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.051745    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/554cfa21-b6f6-44e2-9d05-c46047bb3539-istiod-ca-cert\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.051966 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.051837    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-podinfo\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.051966 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.051885    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-workload-socket\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.051966 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.051931    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-token\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.052222 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.051991    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-envoy\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.052222 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.052021    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-credential-socket\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.052222 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.052121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-workload-certs\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.152652 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.152610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-token\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.152855 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.152665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-envoy\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.152855 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.152685    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-credential-socket\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.152855 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.152729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-workload-certs\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.152855 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.152749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g2s6\" (UniqueName: \"kubernetes.io/projected/554cfa21-b6f6-44e2-9d05-c46047bb3539-kube-api-access-6g2s6\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.152855 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.152780    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-data\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.152855 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.152820    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/554cfa21-b6f6-44e2-9d05-c46047bb3539-istiod-ca-cert\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.152855 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.152851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-podinfo\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.153270 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.152880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-workload-socket\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.153270 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.153192    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-workload-certs\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.153270 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.153256    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-workload-socket\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.153433 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.153311    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-credential-socket\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.153507 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.153479    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-data\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.153686 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.153654    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/554cfa21-b6f6-44e2-9d05-c46047bb3539-istiod-ca-cert\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.155030 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.155007    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-envoy\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.155378 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.155360    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-podinfo\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.160482 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.160457    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-token\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.160591 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.160545    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g2s6\" (UniqueName: \"kubernetes.io/projected/554cfa21-b6f6-44e2-9d05-c46047bb3539-kube-api-access-6g2s6\") pod \"e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") " pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.263552 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.263462    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:31.536377 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.536344    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"]
Sep 07 12:09:31.538968 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:09:31.538944    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod554cfa21_b6f6_44e2_9d05_c46047bb3539.slice/crio-266d58d38f30836e189dc517c73fec1e0cd6ef90cad561d5ffdc4e60af72a224 WatchSource:0}: Error finding container 266d58d38f30836e189dc517c73fec1e0cd6ef90cad561d5ffdc4e60af72a224: Status 404 returned error can't find the container with id 266d58d38f30836e189dc517c73fec1e0cd6ef90cad561d5ffdc4e60af72a224
Sep 07 12:09:31.541729 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.541696    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:31.541819 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.541773    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:31.541893 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.541816    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:31.734531 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.734496    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d" event={"ID":"554cfa21-b6f6-44e2-9d05-c46047bb3539","Type":"ContainerStarted","Data":"47b8f95fd9a81ae3a9b7446c889c1ea1f9fa045ebe05570616d3aa1acba97c3d"}
Sep 07 12:09:31.734983 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.734551    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d" event={"ID":"554cfa21-b6f6-44e2-9d05-c46047bb3539","Type":"ContainerStarted","Data":"266d58d38f30836e189dc517c73fec1e0cd6ef90cad561d5ffdc4e60af72a224"}
Sep 07 12:09:31.757432 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:31.757368    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d" podStartSLOduration=1.7573456950000002 podStartE2EDuration="1.757345695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:09:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:09:31.755108876 +0000 UTC m=+1326.250998302" watchObservedRunningTime="2026-09-07 12:09:31.757345695 +0000 UTC m=+1326.253235119"
Sep 07 12:09:32.264027 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:32.263939    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:32.268607 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:32.268584    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:32.741357 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:32.741322    2580 generic.go:356] "Generic (PLEG): container finished" podID="3077305f-0902-4007-a3e6-beabfbb2a12d" containerID="79050e3f9288b209747c68d170ef93552d85d58f3374f577d5c22f546dfb5bf6" exitCode=0
Sep 07 12:09:32.741768 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:32.741392    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf" event={"ID":"3077305f-0902-4007-a3e6-beabfbb2a12d","Type":"ContainerDied","Data":"79050e3f9288b209747c68d170ef93552d85d58f3374f577d5c22f546dfb5bf6"}
Sep 07 12:09:32.741824 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:32.741770    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:32.741981 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:32.741951    2580 scope.go:117] "RemoveContainer" containerID="79050e3f9288b209747c68d170ef93552d85d58f3374f577d5c22f546dfb5bf6"
Sep 07 12:09:32.742707 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:32.742688    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:32.743063 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:32.743036    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:32.743134 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:32.743119    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:32.743170 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:32.743160    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:33.754099 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:33.754064    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf" event={"ID":"3077305f-0902-4007-a3e6-beabfbb2a12d","Type":"ContainerStarted","Data":"e58febf34d8e6a5136fcf0d494f29185fd11f9c5355e3765c9eeb289ecacfe4b"}
Sep 07 12:09:34.498085 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:34.498047    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:34.498322 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:34.498296    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 07 12:09:34.498379 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:34.498350    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf" podUID="3077305f-0902-4007-a3e6-beabfbb2a12d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 07 12:09:35.498002 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.497963    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 07 12:09:35.498385 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.498023    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf" podUID="3077305f-0902-4007-a3e6-beabfbb2a12d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 07 12:09:35.560664 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.560624    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"]
Sep 07 12:09:35.563818 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.563792    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"
Sep 07 12:09:35.568788 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.567056    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-613886-dockercfg-hv7vm\""
Sep 07 12:09:35.569247 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.569228    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-613886\""
Sep 07 12:09:35.569555 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.569537    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"]
Sep 07 12:09:35.583331 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.583305    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"]
Sep 07 12:09:35.583519 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.583494    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"
Sep 07 12:09:35.583657 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.583508    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"]
Sep 07 12:09:35.587551 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.587524    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73bbe142-c824-48e6-8807-2d5ae9b606f3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n\" (UID: \"73bbe142-c824-48e6-8807-2d5ae9b606f3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"
Sep 07 12:09:35.587679 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.587571    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfpdx\" (UniqueName: \"kubernetes.io/projected/def4ecac-2bb7-4b60-bb71-2a662aeb57fe-kube-api-access-cfpdx\") pod \"payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9\" (UID: \"def4ecac-2bb7-4b60-bb71-2a662aeb57fe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"
Sep 07 12:09:35.587679 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.587609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/def4ecac-2bb7-4b60-bb71-2a662aeb57fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9\" (UID: \"def4ecac-2bb7-4b60-bb71-2a662aeb57fe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"
Sep 07 12:09:35.587829 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.587695    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgpgs\" (UniqueName: \"kubernetes.io/projected/73bbe142-c824-48e6-8807-2d5ae9b606f3-kube-api-access-rgpgs\") pod \"payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n\" (UID: \"73bbe142-c824-48e6-8807-2d5ae9b606f3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"
Sep 07 12:09:35.688489 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.688457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73bbe142-c824-48e6-8807-2d5ae9b606f3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n\" (UID: \"73bbe142-c824-48e6-8807-2d5ae9b606f3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"
Sep 07 12:09:35.688681 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.688530    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfpdx\" (UniqueName: \"kubernetes.io/projected/def4ecac-2bb7-4b60-bb71-2a662aeb57fe-kube-api-access-cfpdx\") pod \"payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9\" (UID: \"def4ecac-2bb7-4b60-bb71-2a662aeb57fe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"
Sep 07 12:09:35.688681 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.688585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/def4ecac-2bb7-4b60-bb71-2a662aeb57fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9\" (UID: \"def4ecac-2bb7-4b60-bb71-2a662aeb57fe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"
Sep 07 12:09:35.688681 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.688636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgpgs\" (UniqueName: \"kubernetes.io/projected/73bbe142-c824-48e6-8807-2d5ae9b606f3-kube-api-access-rgpgs\") pod \"payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n\" (UID: \"73bbe142-c824-48e6-8807-2d5ae9b606f3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"
Sep 07 12:09:35.689349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.689313    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73bbe142-c824-48e6-8807-2d5ae9b606f3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n\" (UID: \"73bbe142-c824-48e6-8807-2d5ae9b606f3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"
Sep 07 12:09:35.689653 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.689629    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/def4ecac-2bb7-4b60-bb71-2a662aeb57fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9\" (UID: \"def4ecac-2bb7-4b60-bb71-2a662aeb57fe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"
Sep 07 12:09:35.697521 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.697480    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgpgs\" (UniqueName: \"kubernetes.io/projected/73bbe142-c824-48e6-8807-2d5ae9b606f3-kube-api-access-rgpgs\") pod \"payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n\" (UID: \"73bbe142-c824-48e6-8807-2d5ae9b606f3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"
Sep 07 12:09:35.698270 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.698248    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfpdx\" (UniqueName: \"kubernetes.io/projected/def4ecac-2bb7-4b60-bb71-2a662aeb57fe-kube-api-access-cfpdx\") pod \"payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9\" (UID: \"def4ecac-2bb7-4b60-bb71-2a662aeb57fe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"
Sep 07 12:09:35.882570 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.882526    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"
Sep 07 12:09:35.895455 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:35.895429    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"
Sep 07 12:09:36.155247 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:36.155148    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"]
Sep 07 12:09:36.197626 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:36.197600    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"]
Sep 07 12:09:36.201236 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:09:36.201189    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73bbe142_c824_48e6_8807_2d5ae9b606f3.slice/crio-d13f9923120a25b2bb39eab4b3c91a838a34a9498206015ca6136073b09d96e0 WatchSource:0}: Error finding container d13f9923120a25b2bb39eab4b3c91a838a34a9498206015ca6136073b09d96e0: Status 404 returned error can't find the container with id d13f9923120a25b2bb39eab4b3c91a838a34a9498206015ca6136073b09d96e0
Sep 07 12:09:36.498758 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:36.498669    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 07 12:09:36.498758 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:36.498730    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf" podUID="3077305f-0902-4007-a3e6-beabfbb2a12d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 07 12:09:36.780173 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:36.780076    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n" event={"ID":"73bbe142-c824-48e6-8807-2d5ae9b606f3","Type":"ContainerStarted","Data":"8b91d926988ce276d0802edb42fa8450a887d3fb4e930e38aef0149b8f8ca271"}
Sep 07 12:09:36.780470 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:36.780154    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n" event={"ID":"73bbe142-c824-48e6-8807-2d5ae9b606f3","Type":"ContainerStarted","Data":"d13f9923120a25b2bb39eab4b3c91a838a34a9498206015ca6136073b09d96e0"}
Sep 07 12:09:36.780544 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:36.780529    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"
Sep 07 12:09:36.782093 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:36.782069    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9" event={"ID":"def4ecac-2bb7-4b60-bb71-2a662aeb57fe","Type":"ContainerStarted","Data":"a0da1daaf1347314fd02ea470ca13eb460bad568f80cc59b02f3a1dab2883335"}
Sep 07 12:09:36.782228 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:36.782115    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9" event={"ID":"def4ecac-2bb7-4b60-bb71-2a662aeb57fe","Type":"ContainerStarted","Data":"e457be51cfb488af77714597c363d19958c64c014c5a5404b67e96ff13c1fdce"}
Sep 07 12:09:36.782228 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:36.782146    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"
Sep 07 12:09:36.800883 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:36.800832    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n" podStartSLOduration=1.80081298 podStartE2EDuration="1.80081298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:09:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:09:36.79768691 +0000 UTC m=+1331.293576324" watchObservedRunningTime="2026-09-07 12:09:36.80081298 +0000 UTC m=+1331.296702405"
Sep 07 12:09:36.824308 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:36.824256    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9" podStartSLOduration=1.824241676 podStartE2EDuration="1.824241676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:09:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:09:36.823740593 +0000 UTC m=+1331.319630018" watchObservedRunningTime="2026-09-07 12:09:36.824241676 +0000 UTC m=+1331.320131099"
Sep 07 12:09:37.499037 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:37.498999    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 07 12:09:37.499448 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:37.499068    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf" podUID="3077305f-0902-4007-a3e6-beabfbb2a12d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 07 12:09:38.498537 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:38.498492    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 07 12:09:38.498742 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:38.498559    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf" podUID="3077305f-0902-4007-a3e6-beabfbb2a12d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 07 12:09:39.721103 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:39.721068    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:39.721757 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:39.721730    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:39.724095 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:39.724018    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:09:45.731253 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:45.728660    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"]
Sep 07 12:09:45.733777 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:45.733744    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"
Sep 07 12:09:45.736449 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:45.736428    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-bff0c9-dockercfg-hnl4c\""
Sep 07 12:09:45.736589 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:45.736466    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-bff0c9\""
Sep 07 12:09:45.741691 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:45.741658    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"]
Sep 07 12:09:45.761222 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:45.759894    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"]
Sep 07 12:09:45.779539 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:45.779504    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"
Sep 07 12:09:45.781354 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:45.781330    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"]
Sep 07 12:09:45.800555 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:45.800516    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f219ca8d-1c59-48bc-a256-961c65ab899f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686\" (UID: \"f219ca8d-1c59-48bc-a256-961c65ab899f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"
Sep 07 12:09:45.800706 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:45.800561    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs6jv\" (UniqueName: \"kubernetes.io/projected/f219ca8d-1c59-48bc-a256-961c65ab899f-kube-api-access-bs6jv\") pod \"payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686\" (UID: \"f219ca8d-1c59-48bc-a256-961c65ab899f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"
Sep 07 12:09:45.902086 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:45.902055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zltw\" (UniqueName: \"kubernetes.io/projected/f1d5239c-d80b-4951-952a-c09711a123f3-kube-api-access-9zltw\") pod \"payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27\" (UID: \"f1d5239c-d80b-4951-952a-c09711a123f3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"
Sep 07 12:09:45.902378 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:45.902363    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f219ca8d-1c59-48bc-a256-961c65ab899f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686\" (UID: \"f219ca8d-1c59-48bc-a256-961c65ab899f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"
Sep 07 12:09:45.903130 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:45.903105    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs6jv\" (UniqueName: \"kubernetes.io/projected/f219ca8d-1c59-48bc-a256-961c65ab899f-kube-api-access-bs6jv\") pod \"payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686\" (UID: \"f219ca8d-1c59-48bc-a256-961c65ab899f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"
Sep 07 12:09:45.903322 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:45.903308    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1d5239c-d80b-4951-952a-c09711a123f3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27\" (UID: \"f1d5239c-d80b-4951-952a-c09711a123f3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"
Sep 07 12:09:45.903436 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:45.903048    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f219ca8d-1c59-48bc-a256-961c65ab899f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686\" (UID: \"f219ca8d-1c59-48bc-a256-961c65ab899f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"
Sep 07 12:09:45.910927 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:45.910904    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs6jv\" (UniqueName: \"kubernetes.io/projected/f219ca8d-1c59-48bc-a256-961c65ab899f-kube-api-access-bs6jv\") pod \"payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686\" (UID: \"f219ca8d-1c59-48bc-a256-961c65ab899f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"
Sep 07 12:09:46.005090 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:46.004987    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1d5239c-d80b-4951-952a-c09711a123f3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27\" (UID: \"f1d5239c-d80b-4951-952a-c09711a123f3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"
Sep 07 12:09:46.005318 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:46.005110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zltw\" (UniqueName: \"kubernetes.io/projected/f1d5239c-d80b-4951-952a-c09711a123f3-kube-api-access-9zltw\") pod \"payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27\" (UID: \"f1d5239c-d80b-4951-952a-c09711a123f3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"
Sep 07 12:09:46.005764 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:46.005743    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1d5239c-d80b-4951-952a-c09711a123f3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27\" (UID: \"f1d5239c-d80b-4951-952a-c09711a123f3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"
Sep 07 12:09:46.013349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:46.013319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zltw\" (UniqueName: \"kubernetes.io/projected/f1d5239c-d80b-4951-952a-c09711a123f3-kube-api-access-9zltw\") pod \"payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27\" (UID: \"f1d5239c-d80b-4951-952a-c09711a123f3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"
Sep 07 12:09:46.047073 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:46.047031    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"
Sep 07 12:09:46.090004 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:46.089933    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"
Sep 07 12:09:46.335318 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:46.335294    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"]
Sep 07 12:09:46.337189 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:09:46.337163    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf219ca8d_1c59_48bc_a256_961c65ab899f.slice/crio-82694e014f39b523fa5c082685a7e81803bfaa11035ed3b36f2ecebf2a37e6cb WatchSource:0}: Error finding container 82694e014f39b523fa5c082685a7e81803bfaa11035ed3b36f2ecebf2a37e6cb: Status 404 returned error can't find the container with id 82694e014f39b523fa5c082685a7e81803bfaa11035ed3b36f2ecebf2a37e6cb
Sep 07 12:09:46.394063 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:46.394038    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"]
Sep 07 12:09:46.398365 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:09:46.398340    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1d5239c_d80b_4951_952a_c09711a123f3.slice/crio-e0b0653ed6a7abf0a465ba6f7af9e114d5772d4d330b250a53201a862e49d0af WatchSource:0}: Error finding container e0b0653ed6a7abf0a465ba6f7af9e114d5772d4d330b250a53201a862e49d0af: Status 404 returned error can't find the container with id e0b0653ed6a7abf0a465ba6f7af9e114d5772d4d330b250a53201a862e49d0af
Sep 07 12:09:46.835425 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:46.835387    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27" event={"ID":"f1d5239c-d80b-4951-952a-c09711a123f3","Type":"ContainerStarted","Data":"2870fc85e55e19af8d1fce5aad3470c31eb9d1aa109fd8d990d2b9a8afd00e8e"}
Sep 07 12:09:46.835923 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:46.835450    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27" event={"ID":"f1d5239c-d80b-4951-952a-c09711a123f3","Type":"ContainerStarted","Data":"e0b0653ed6a7abf0a465ba6f7af9e114d5772d4d330b250a53201a862e49d0af"}
Sep 07 12:09:46.835923 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:46.835505    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"
Sep 07 12:09:46.837117 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:46.837095    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686" event={"ID":"f219ca8d-1c59-48bc-a256-961c65ab899f","Type":"ContainerStarted","Data":"a3b881b0fd4c62d45ec1f0fa90369bc884c14bb7daf2060e52a1affac2a26393"}
Sep 07 12:09:46.837247 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:46.837138    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686" event={"ID":"f219ca8d-1c59-48bc-a256-961c65ab899f","Type":"ContainerStarted","Data":"82694e014f39b523fa5c082685a7e81803bfaa11035ed3b36f2ecebf2a37e6cb"}
Sep 07 12:09:46.837247 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:46.837236    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"
Sep 07 12:09:46.872312 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:46.872241    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27" podStartSLOduration=1.872216593 podStartE2EDuration="1.872216593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:09:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:09:46.856176922 +0000 UTC m=+1341.352066347" watchObservedRunningTime="2026-09-07 12:09:46.872216593 +0000 UTC m=+1341.368106013"
Sep 07 12:09:46.873897 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:46.873843    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686" podStartSLOduration=1.8738292159999999 podStartE2EDuration="1.873829216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:09:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:09:46.871338657 +0000 UTC m=+1341.367228082" watchObservedRunningTime="2026-09-07 12:09:46.873829216 +0000 UTC m=+1341.369718640"
Sep 07 12:09:47.790837 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:47.790805    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"
Sep 07 12:09:47.791038 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:47.790890    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"
Sep 07 12:09:48.784404 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:48.784358    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 07 12:09:48.784810 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:48.784461    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d" podUID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 07 12:09:50.863361 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:50.863317    2580 generic.go:356] "Generic (PLEG): container finished" podID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerID="47b8f95fd9a81ae3a9b7446c889c1ea1f9fa045ebe05570616d3aa1acba97c3d" exitCode=0
Sep 07 12:09:50.863810 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:50.863392    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d" event={"ID":"554cfa21-b6f6-44e2-9d05-c46047bb3539","Type":"ContainerDied","Data":"47b8f95fd9a81ae3a9b7446c889c1ea1f9fa045ebe05570616d3aa1acba97c3d"}
Sep 07 12:09:50.863810 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:50.863756    2580 scope.go:117] "RemoveContainer" containerID="47b8f95fd9a81ae3a9b7446c889c1ea1f9fa045ebe05570616d3aa1acba97c3d"
Sep 07 12:09:50.864801 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:50.864772    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:50.864917 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:50.864854    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:50.864917 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:50.864902    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 07 12:09:51.876852 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:51.876805    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d" event={"ID":"554cfa21-b6f6-44e2-9d05-c46047bb3539","Type":"ContainerStarted","Data":"a30560e2000e8a8fabe11a52da4d2617c7296e0bb82b396966fdaf0183e28c7d"}
Sep 07 12:09:52.264370 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:52.264265    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:52.264632 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:52.264345    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 07 12:09:52.264781 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:52.264758    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d" podUID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 07 12:09:53.264490 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:53.264450    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 07 12:09:53.264979 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:53.264533    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d" podUID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 07 12:09:54.264082 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:54.264031    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 07 12:09:54.264322 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:54.264105    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d" podUID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 07 12:09:55.264134 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:55.264082    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 07 12:09:55.264664 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:55.264161    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d" podUID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 07 12:09:56.266074 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:56.266032    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 07 12:09:56.267058 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:56.266657    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d" podUID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 07 12:09:57.264455 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:57.264417    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 07 12:09:57.264631 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:57.264478    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d" podUID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 07 12:09:57.847350 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:57.847319    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"
Sep 07 12:09:57.847747 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:57.847569    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"
Sep 07 12:09:58.267911 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:58.267879    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:58.268179 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:58.268157    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:09:58.268822 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:09:58.268806    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:10:28.016034 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:28.015996    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"]
Sep 07 12:10:28.019060 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:28.019039    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"
Sep 07 12:10:28.021783 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:28.021762    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-619f5b7c\""
Sep 07 12:10:28.021887 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:28.021764    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-619f5b7c-dockercfg-mmbr4\""
Sep 07 12:10:28.029897 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:28.029874    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"]
Sep 07 12:10:28.133581 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:28.133535    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prflh\" (UniqueName: \"kubernetes.io/projected/a0619b33-d5ed-4c3b-8a50-663918cbec60-kube-api-access-prflh\") pod \"payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg\" (UID: \"a0619b33-d5ed-4c3b-8a50-663918cbec60\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"
Sep 07 12:10:28.133793 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:28.133622    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0619b33-d5ed-4c3b-8a50-663918cbec60-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg\" (UID: \"a0619b33-d5ed-4c3b-8a50-663918cbec60\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"
Sep 07 12:10:28.235046 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:28.235011    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prflh\" (UniqueName: \"kubernetes.io/projected/a0619b33-d5ed-4c3b-8a50-663918cbec60-kube-api-access-prflh\") pod \"payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg\" (UID: \"a0619b33-d5ed-4c3b-8a50-663918cbec60\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"
Sep 07 12:10:28.235260 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:28.235059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0619b33-d5ed-4c3b-8a50-663918cbec60-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg\" (UID: \"a0619b33-d5ed-4c3b-8a50-663918cbec60\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"
Sep 07 12:10:28.235749 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:28.235729    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0619b33-d5ed-4c3b-8a50-663918cbec60-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg\" (UID: \"a0619b33-d5ed-4c3b-8a50-663918cbec60\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"
Sep 07 12:10:28.242791 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:28.242765    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prflh\" (UniqueName: \"kubernetes.io/projected/a0619b33-d5ed-4c3b-8a50-663918cbec60-kube-api-access-prflh\") pod \"payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg\" (UID: \"a0619b33-d5ed-4c3b-8a50-663918cbec60\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"
Sep 07 12:10:28.330189 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:28.330096    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"
Sep 07 12:10:28.603924 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:28.603860    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"]
Sep 07 12:10:28.606128 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:10:28.606092    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0619b33_d5ed_4c3b_8a50_663918cbec60.slice/crio-b2282b299e498cc132e1acbb15b8af189ffb8ea394f3ac6e55316ba90daa5ad3 WatchSource:0}: Error finding container b2282b299e498cc132e1acbb15b8af189ffb8ea394f3ac6e55316ba90daa5ad3: Status 404 returned error can't find the container with id b2282b299e498cc132e1acbb15b8af189ffb8ea394f3ac6e55316ba90daa5ad3
Sep 07 12:10:29.071163 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:29.071115    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg" event={"ID":"a0619b33-d5ed-4c3b-8a50-663918cbec60","Type":"ContainerStarted","Data":"015ca3228e821a3333630d784adcb9ae46f922636b8d407958ea1b8bf33798ba"}
Sep 07 12:10:29.071611 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:29.071184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg" event={"ID":"a0619b33-d5ed-4c3b-8a50-663918cbec60","Type":"ContainerStarted","Data":"b2282b299e498cc132e1acbb15b8af189ffb8ea394f3ac6e55316ba90daa5ad3"}
Sep 07 12:10:29.071611 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:29.071225    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"
Sep 07 12:10:29.087914 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:29.087871    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg" podStartSLOduration=2.087857469 podStartE2EDuration="2.087857469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:10:29.085431209 +0000 UTC m=+1383.581320652" watchObservedRunningTime="2026-09-07 12:10:29.087857469 +0000 UTC m=+1383.583746891"
Sep 07 12:10:40.077923 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:10:40.077890    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"
Sep 07 12:11:26.127393 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:11:26.127317    2580 scope.go:117] "RemoveContainer" containerID="5bba5e1e3a80ad4c0ab578ba6342d6e91a7a07eb8f696dad03900478801d73f3"
Sep 07 12:11:26.136003 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:11:26.135983    2580 scope.go:117] "RemoveContainer" containerID="7a1522db2404a5e874590de93ff7f84422492cf92541ef4aef73d61e6f7ed2df"
Sep 07 12:12:01.125160 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:01.125122    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt"]
Sep 07 12:12:01.128654 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:01.128634    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt"
Sep 07 12:12:01.131228 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:01.131181    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:12:01.131228 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:01.131182    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:12:01.134912 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:01.134875    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt"]
Sep 07 12:12:01.217601 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:01.217562    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/692a6ec2-83d6-4215-a057-cf765a819e0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt\" (UID: \"692a6ec2-83d6-4215-a057-cf765a819e0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt"
Sep 07 12:12:01.318370 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:01.318338    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/692a6ec2-83d6-4215-a057-cf765a819e0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt\" (UID: \"692a6ec2-83d6-4215-a057-cf765a819e0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt"
Sep 07 12:12:01.318953 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:01.318934    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/692a6ec2-83d6-4215-a057-cf765a819e0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt\" (UID: \"692a6ec2-83d6-4215-a057-cf765a819e0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt"
Sep 07 12:12:01.440174 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:01.440081    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt"
Sep 07 12:12:01.708721 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:01.708693    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt"]
Sep 07 12:12:01.711329 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:12:01.711301    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod692a6ec2_83d6_4215_a057_cf765a819e0d.slice/crio-5e82b42f0c0ef3f0f010741b23a1a121b221f8f6900618b8cde3e7153ba18cb8 WatchSource:0}: Error finding container 5e82b42f0c0ef3f0f010741b23a1a121b221f8f6900618b8cde3e7153ba18cb8: Status 404 returned error can't find the container with id 5e82b42f0c0ef3f0f010741b23a1a121b221f8f6900618b8cde3e7153ba18cb8
Sep 07 12:12:02.549424 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:02.549340    2580 generic.go:356] "Generic (PLEG): container finished" podID="692a6ec2-83d6-4215-a057-cf765a819e0d" containerID="7df2555669fb8a11ee053c705ff949a5b50d2580742487030e7bd80ddf3a2ece" exitCode=0
Sep 07 12:12:02.549795 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:02.549423    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt" event={"ID":"692a6ec2-83d6-4215-a057-cf765a819e0d","Type":"ContainerDied","Data":"7df2555669fb8a11ee053c705ff949a5b50d2580742487030e7bd80ddf3a2ece"}
Sep 07 12:12:02.549795 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:02.549481    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt" event={"ID":"692a6ec2-83d6-4215-a057-cf765a819e0d","Type":"ContainerStarted","Data":"5e82b42f0c0ef3f0f010741b23a1a121b221f8f6900618b8cde3e7153ba18cb8"}
Sep 07 12:12:03.689039 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:03.689015    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt"
Sep 07 12:12:03.743045 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:03.743013    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/692a6ec2-83d6-4215-a057-cf765a819e0d-maas-api-service-ca\") pod \"692a6ec2-83d6-4215-a057-cf765a819e0d\" (UID: \"692a6ec2-83d6-4215-a057-cf765a819e0d\") "
Sep 07 12:12:03.743415 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:03.743393    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/692a6ec2-83d6-4215-a057-cf765a819e0d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "692a6ec2-83d6-4215-a057-cf765a819e0d" (UID: "692a6ec2-83d6-4215-a057-cf765a819e0d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:12:03.844873 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:03.844777    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/692a6ec2-83d6-4215-a057-cf765a819e0d-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:12:04.561799 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:04.561764    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt"
Sep 07 12:12:04.561799 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:04.561774    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt" event={"ID":"692a6ec2-83d6-4215-a057-cf765a819e0d","Type":"ContainerDied","Data":"5e82b42f0c0ef3f0f010741b23a1a121b221f8f6900618b8cde3e7153ba18cb8"}
Sep 07 12:12:04.562024 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:04.561823    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e82b42f0c0ef3f0f010741b23a1a121b221f8f6900618b8cde3e7153ba18cb8"
Sep 07 12:12:16.695282 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:16.695247    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"]
Sep 07 12:12:16.695703 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:16.695532    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg" podUID="a0619b33-d5ed-4c3b-8a50-663918cbec60" containerName="payload-pre-processing" containerID="cri-o://015ca3228e821a3333630d784adcb9ae46f922636b8d407958ea1b8bf33798ba" gracePeriod=30
Sep 07 12:12:16.971266 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:16.971230    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt"]
Sep 07 12:12:16.972663 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:16.972640    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-619f5b7c-6x2qt"]
Sep 07 12:12:16.985304 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:16.985279    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"
Sep 07 12:12:17.075758 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:17.075724    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0619b33-d5ed-4c3b-8a50-663918cbec60-plugins-config-volume\") pod \"a0619b33-d5ed-4c3b-8a50-663918cbec60\" (UID: \"a0619b33-d5ed-4c3b-8a50-663918cbec60\") "
Sep 07 12:12:17.075943 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:17.075809    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prflh\" (UniqueName: \"kubernetes.io/projected/a0619b33-d5ed-4c3b-8a50-663918cbec60-kube-api-access-prflh\") pod \"a0619b33-d5ed-4c3b-8a50-663918cbec60\" (UID: \"a0619b33-d5ed-4c3b-8a50-663918cbec60\") "
Sep 07 12:12:17.076171 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:17.076133    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0619b33-d5ed-4c3b-8a50-663918cbec60-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a0619b33-d5ed-4c3b-8a50-663918cbec60" (UID: "a0619b33-d5ed-4c3b-8a50-663918cbec60"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:12:17.078008 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:17.077976    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0619b33-d5ed-4c3b-8a50-663918cbec60-kube-api-access-prflh" (OuterVolumeSpecName: "kube-api-access-prflh") pod "a0619b33-d5ed-4c3b-8a50-663918cbec60" (UID: "a0619b33-d5ed-4c3b-8a50-663918cbec60"). InnerVolumeSpecName "kube-api-access-prflh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:12:17.177489 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:17.177454    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0619b33-d5ed-4c3b-8a50-663918cbec60-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:12:17.177489 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:17.177483    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prflh\" (UniqueName: \"kubernetes.io/projected/a0619b33-d5ed-4c3b-8a50-663918cbec60-kube-api-access-prflh\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:12:17.628698 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:17.628657    2580 generic.go:356] "Generic (PLEG): container finished" podID="a0619b33-d5ed-4c3b-8a50-663918cbec60" containerID="015ca3228e821a3333630d784adcb9ae46f922636b8d407958ea1b8bf33798ba" exitCode=0
Sep 07 12:12:17.628880 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:17.628706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg" event={"ID":"a0619b33-d5ed-4c3b-8a50-663918cbec60","Type":"ContainerDied","Data":"015ca3228e821a3333630d784adcb9ae46f922636b8d407958ea1b8bf33798ba"}
Sep 07 12:12:17.628880 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:17.628731    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"
Sep 07 12:12:17.628880 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:17.628788    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg" event={"ID":"a0619b33-d5ed-4c3b-8a50-663918cbec60","Type":"ContainerDied","Data":"b2282b299e498cc132e1acbb15b8af189ffb8ea394f3ac6e55316ba90daa5ad3"}
Sep 07 12:12:17.628880 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:17.628819    2580 scope.go:117] "RemoveContainer" containerID="015ca3228e821a3333630d784adcb9ae46f922636b8d407958ea1b8bf33798ba"
Sep 07 12:12:17.638166 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:17.638139    2580 scope.go:117] "RemoveContainer" containerID="015ca3228e821a3333630d784adcb9ae46f922636b8d407958ea1b8bf33798ba"
Sep 07 12:12:17.638510 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:12:17.638487    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"015ca3228e821a3333630d784adcb9ae46f922636b8d407958ea1b8bf33798ba\": container with ID starting with 015ca3228e821a3333630d784adcb9ae46f922636b8d407958ea1b8bf33798ba not found: ID does not exist" containerID="015ca3228e821a3333630d784adcb9ae46f922636b8d407958ea1b8bf33798ba"
Sep 07 12:12:17.638615 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:17.638519    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"015ca3228e821a3333630d784adcb9ae46f922636b8d407958ea1b8bf33798ba"} err="failed to get container status \"015ca3228e821a3333630d784adcb9ae46f922636b8d407958ea1b8bf33798ba\": rpc error: code = NotFound desc = could not find container \"015ca3228e821a3333630d784adcb9ae46f922636b8d407958ea1b8bf33798ba\": container with ID starting with 015ca3228e821a3333630d784adcb9ae46f922636b8d407958ea1b8bf33798ba not found: ID does not exist"
Sep 07 12:12:17.649484 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:17.649461    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"]
Sep 07 12:12:17.653078 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:17.653040    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-619f5b7c-789ccdbbbb-v7wqg"]
Sep 07 12:12:18.010599 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:18.010522    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="692a6ec2-83d6-4215-a057-cf765a819e0d" path="/var/lib/kubelet/pods/692a6ec2-83d6-4215-a057-cf765a819e0d/volumes"
Sep 07 12:12:18.010939 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:18.010886    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0619b33-d5ed-4c3b-8a50-663918cbec60" path="/var/lib/kubelet/pods/a0619b33-d5ed-4c3b-8a50-663918cbec60/volumes"
Sep 07 12:12:26.158163 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:26.158128    2580 scope.go:117] "RemoveContainer" containerID="23815a1fcf8938a88ff9b54b75c290da4c17b164d52fd77d9f80bbbdef5f621a"
Sep 07 12:12:26.166337 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:26.166320    2580 scope.go:117] "RemoveContainer" containerID="0ff232ed2d4b5c37b01d34f5df5d89a5245c9f4b51a3ddb1014d8583b0cebd91"
Sep 07 12:12:26.173653 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:12:26.173632    2580 scope.go:117] "RemoveContainer" containerID="cc0897b76854629d99d7a7bc1b2f24ba658ba8abb453445a8bad49765708abe7"
Sep 07 12:13:26.201524 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:13:26.201452    2580 scope.go:117] "RemoveContainer" containerID="39edbc89e789b1524964eb967c530e9bc99486fe86f024eed29da99f6b12959e"
Sep 07 12:14:13.052280 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.052242    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"]
Sep 07 12:14:13.052735 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.052607    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="692a6ec2-83d6-4215-a057-cf765a819e0d" containerName="revoke-keys"
Sep 07 12:14:13.052735 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.052618    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="692a6ec2-83d6-4215-a057-cf765a819e0d" containerName="revoke-keys"
Sep 07 12:14:13.052735 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.052630    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0619b33-d5ed-4c3b-8a50-663918cbec60" containerName="payload-pre-processing"
Sep 07 12:14:13.052735 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.052636    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0619b33-d5ed-4c3b-8a50-663918cbec60" containerName="payload-pre-processing"
Sep 07 12:14:13.052735 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.052699    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0619b33-d5ed-4c3b-8a50-663918cbec60" containerName="payload-pre-processing"
Sep 07 12:14:13.052735 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.052709    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="692a6ec2-83d6-4215-a057-cf765a819e0d" containerName="revoke-keys"
Sep 07 12:14:13.055805 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.055787    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"
Sep 07 12:14:13.058221 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.058181    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-f7f063-dockercfg-vbrrm\""
Sep 07 12:14:13.058221 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.058191    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-f7f063\""
Sep 07 12:14:13.066037 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.065972    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"]
Sep 07 12:14:13.086811 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.086772    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"]
Sep 07 12:14:13.090866 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.090836    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"
Sep 07 12:14:13.102808 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.102775    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"]
Sep 07 12:14:13.121133 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.121097    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trk2r\" (UniqueName: \"kubernetes.io/projected/a6b1e0f2-583a-4a70-af8a-337ec354424f-kube-api-access-trk2r\") pod \"payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck\" (UID: \"a6b1e0f2-583a-4a70-af8a-337ec354424f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"
Sep 07 12:14:13.121330 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.121148    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6b1e0f2-583a-4a70-af8a-337ec354424f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck\" (UID: \"a6b1e0f2-583a-4a70-af8a-337ec354424f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"
Sep 07 12:14:13.221915 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.221868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6b1e0f2-583a-4a70-af8a-337ec354424f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck\" (UID: \"a6b1e0f2-583a-4a70-af8a-337ec354424f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"
Sep 07 12:14:13.222157 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.221927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld82m\" (UniqueName: \"kubernetes.io/projected/89c1084a-15ca-456b-a8e5-5b0eedd872eb-kube-api-access-ld82m\") pod \"payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv\" (UID: \"89c1084a-15ca-456b-a8e5-5b0eedd872eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"
Sep 07 12:14:13.222157 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.221957    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89c1084a-15ca-456b-a8e5-5b0eedd872eb-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv\" (UID: \"89c1084a-15ca-456b-a8e5-5b0eedd872eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"
Sep 07 12:14:13.222157 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.222115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trk2r\" (UniqueName: \"kubernetes.io/projected/a6b1e0f2-583a-4a70-af8a-337ec354424f-kube-api-access-trk2r\") pod \"payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck\" (UID: \"a6b1e0f2-583a-4a70-af8a-337ec354424f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"
Sep 07 12:14:13.223296 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.223266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6b1e0f2-583a-4a70-af8a-337ec354424f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck\" (UID: \"a6b1e0f2-583a-4a70-af8a-337ec354424f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"
Sep 07 12:14:13.231683 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.231644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trk2r\" (UniqueName: \"kubernetes.io/projected/a6b1e0f2-583a-4a70-af8a-337ec354424f-kube-api-access-trk2r\") pod \"payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck\" (UID: \"a6b1e0f2-583a-4a70-af8a-337ec354424f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"
Sep 07 12:14:13.323132 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.323036    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld82m\" (UniqueName: \"kubernetes.io/projected/89c1084a-15ca-456b-a8e5-5b0eedd872eb-kube-api-access-ld82m\") pod \"payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv\" (UID: \"89c1084a-15ca-456b-a8e5-5b0eedd872eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"
Sep 07 12:14:13.323132 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.323078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89c1084a-15ca-456b-a8e5-5b0eedd872eb-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv\" (UID: \"89c1084a-15ca-456b-a8e5-5b0eedd872eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"
Sep 07 12:14:13.323840 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.323817    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89c1084a-15ca-456b-a8e5-5b0eedd872eb-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv\" (UID: \"89c1084a-15ca-456b-a8e5-5b0eedd872eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"
Sep 07 12:14:13.331214 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.331173    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld82m\" (UniqueName: \"kubernetes.io/projected/89c1084a-15ca-456b-a8e5-5b0eedd872eb-kube-api-access-ld82m\") pod \"payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv\" (UID: \"89c1084a-15ca-456b-a8e5-5b0eedd872eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"
Sep 07 12:14:13.368113 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.368068    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"
Sep 07 12:14:13.403959 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.403920    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"
Sep 07 12:14:13.653322 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.653287    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"]
Sep 07 12:14:13.656804 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:14:13.656768    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6b1e0f2_583a_4a70_af8a_337ec354424f.slice/crio-03e5dda375a7fb4e77a4ddbe059962e8baa601003f785c77cc9044590e5826d4 WatchSource:0}: Error finding container 03e5dda375a7fb4e77a4ddbe059962e8baa601003f785c77cc9044590e5826d4: Status 404 returned error can't find the container with id 03e5dda375a7fb4e77a4ddbe059962e8baa601003f785c77cc9044590e5826d4
Sep 07 12:14:13.689404 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:13.689379    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"]
Sep 07 12:14:13.695297 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:14:13.695273    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89c1084a_15ca_456b_a8e5_5b0eedd872eb.slice/crio-db014052dabe28f21aa426cce3c37a8351676353fb47cdbe6f5d269eab81789f WatchSource:0}: Error finding container db014052dabe28f21aa426cce3c37a8351676353fb47cdbe6f5d269eab81789f: Status 404 returned error can't find the container with id db014052dabe28f21aa426cce3c37a8351676353fb47cdbe6f5d269eab81789f
Sep 07 12:14:14.224092 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:14.224052    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck" event={"ID":"a6b1e0f2-583a-4a70-af8a-337ec354424f","Type":"ContainerStarted","Data":"ca3245d3dbe29a67d5e4dc5b5456ffa1f324afdaac27c1747f206f6a1042b343"}
Sep 07 12:14:14.224545 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:14.224122    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"
Sep 07 12:14:14.224545 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:14.224145    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck" event={"ID":"a6b1e0f2-583a-4a70-af8a-337ec354424f","Type":"ContainerStarted","Data":"03e5dda375a7fb4e77a4ddbe059962e8baa601003f785c77cc9044590e5826d4"}
Sep 07 12:14:14.225746 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:14.225716    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv" event={"ID":"89c1084a-15ca-456b-a8e5-5b0eedd872eb","Type":"ContainerStarted","Data":"d5a6ee99def65568871ba52a7b636b8bdcdaf7a20825d6a28e5ecc979e3eb360"}
Sep 07 12:14:14.225852 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:14.225766    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv" event={"ID":"89c1084a-15ca-456b-a8e5-5b0eedd872eb","Type":"ContainerStarted","Data":"db014052dabe28f21aa426cce3c37a8351676353fb47cdbe6f5d269eab81789f"}
Sep 07 12:14:14.225852 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:14.225843    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"
Sep 07 12:14:14.239489 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:14.239435    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck" podStartSLOduration=1.239416715 podStartE2EDuration="1.239416715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:14:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:14:14.238365339 +0000 UTC m=+1608.734254762" watchObservedRunningTime="2026-09-07 12:14:14.239416715 +0000 UTC m=+1608.735306143"
Sep 07 12:14:14.255589 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:14.255521    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv" podStartSLOduration=1.255495981 podStartE2EDuration="1.255495981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:14:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:14:14.252386476 +0000 UTC m=+1608.748275899" watchObservedRunningTime="2026-09-07 12:14:14.255495981 +0000 UTC m=+1608.751385407"
Sep 07 12:14:25.234964 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:25.234886    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"
Sep 07 12:14:25.234964 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:25.234958    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"
Sep 07 12:14:42.328879 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:42.328842    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"]
Sep 07 12:14:42.332420 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:42.332399    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"
Sep 07 12:14:42.334901 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:42.334876    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-9e422b\""
Sep 07 12:14:42.335043 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:42.334876    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-9e422b-dockercfg-459v6\""
Sep 07 12:14:42.344538 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:42.344513    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"]
Sep 07 12:14:42.404567 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:42.404527    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07aa3fa4-debd-4518-9d50-da879148cee8-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx\" (UID: \"07aa3fa4-debd-4518-9d50-da879148cee8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"
Sep 07 12:14:42.404567 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:42.404569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbc7p\" (UniqueName: \"kubernetes.io/projected/07aa3fa4-debd-4518-9d50-da879148cee8-kube-api-access-rbc7p\") pod \"payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx\" (UID: \"07aa3fa4-debd-4518-9d50-da879148cee8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"
Sep 07 12:14:42.505630 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:42.505591    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07aa3fa4-debd-4518-9d50-da879148cee8-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx\" (UID: \"07aa3fa4-debd-4518-9d50-da879148cee8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"
Sep 07 12:14:42.505630 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:42.505633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbc7p\" (UniqueName: \"kubernetes.io/projected/07aa3fa4-debd-4518-9d50-da879148cee8-kube-api-access-rbc7p\") pod \"payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx\" (UID: \"07aa3fa4-debd-4518-9d50-da879148cee8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"
Sep 07 12:14:42.506352 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:42.506331    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07aa3fa4-debd-4518-9d50-da879148cee8-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx\" (UID: \"07aa3fa4-debd-4518-9d50-da879148cee8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"
Sep 07 12:14:42.516771 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:42.516743    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbc7p\" (UniqueName: \"kubernetes.io/projected/07aa3fa4-debd-4518-9d50-da879148cee8-kube-api-access-rbc7p\") pod \"payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx\" (UID: \"07aa3fa4-debd-4518-9d50-da879148cee8\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"
Sep 07 12:14:42.644889 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:42.644860    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"
Sep 07 12:14:42.916237 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:42.916150    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"]
Sep 07 12:14:42.918526 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:14:42.918503    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07aa3fa4_debd_4518_9d50_da879148cee8.slice/crio-c09b5af3f712f2a2b9c72eaa729b0e34ae8f120e0d89a164dec8519f51b304e0 WatchSource:0}: Error finding container c09b5af3f712f2a2b9c72eaa729b0e34ae8f120e0d89a164dec8519f51b304e0: Status 404 returned error can't find the container with id c09b5af3f712f2a2b9c72eaa729b0e34ae8f120e0d89a164dec8519f51b304e0
Sep 07 12:14:43.372056 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:43.372015    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx" event={"ID":"07aa3fa4-debd-4518-9d50-da879148cee8","Type":"ContainerStarted","Data":"ab31d77ccf8aecdb86aa74e9e258e47aba955c723cb154aedfc43fa2c132a069"}
Sep 07 12:14:43.372480 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:43.372071    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx" event={"ID":"07aa3fa4-debd-4518-9d50-da879148cee8","Type":"ContainerStarted","Data":"c09b5af3f712f2a2b9c72eaa729b0e34ae8f120e0d89a164dec8519f51b304e0"}
Sep 07 12:14:43.372480 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:43.372093    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"
Sep 07 12:14:43.388866 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:43.388819    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx" podStartSLOduration=1.38880372 podStartE2EDuration="1.38880372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:14:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:14:43.386903711 +0000 UTC m=+1637.882793135" watchObservedRunningTime="2026-09-07 12:14:43.38880372 +0000 UTC m=+1637.884693143"
Sep 07 12:14:54.380382 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:14:54.380345    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"
Sep 07 12:15:00.138902 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.138863    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3a0bb2-29813055-txszf"]
Sep 07 12:15:00.143292 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.143267    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bff0c9-29813055-tvgdv"]
Sep 07 12:15:00.143465 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.143400    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3a0bb2-29813055-txszf"
Sep 07 12:15:00.146084 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.146060    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:15:00.147132 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.147114    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-613886-29813055-8pk62"]
Sep 07 12:15:00.147259 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.147244    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bff0c9-29813055-tvgdv"
Sep 07 12:15:00.150518 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.150488    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-9e422b-29813055-7stbn"]
Sep 07 12:15:00.150673 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.150652    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-613886-29813055-8pk62"
Sep 07 12:15:00.158319 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.158286    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3a0bb2-29813055-txszf"
Sep 07 12:15:00.159184 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.159161    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3a0bb2-29813055-txszf"]
Sep 07 12:15:00.159507 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.159492    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bff0c9-29813055-tvgdv"]
Sep 07 12:15:00.159650 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.159637    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-613886-29813055-8pk62"]
Sep 07 12:15:00.159838 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.159826    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-9e422b-29813055-7stbn"
Sep 07 12:15:00.161987 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.161966    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-9e422b-29813055-7stbn"]
Sep 07 12:15:00.167728 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.167706    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f7f063-29813055-lf4kl"]
Sep 07 12:15:00.171743 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.171717    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bff0c9-29813055-tvgdv"
Sep 07 12:15:00.186745 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.186716    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f7f063-29813055-lf4kl"
Sep 07 12:15:00.187860 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.187831    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f7f063-29813055-lf4kl"]
Sep 07 12:15:00.223571 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.222979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-613886-29813055-8pk62"
Sep 07 12:15:00.237151 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.237013    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-9e422b-29813055-7stbn"
Sep 07 12:15:00.276620 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.276572    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f7f063-29813055-lf4kl"
Sep 07 12:15:00.460543 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.460512    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3a0bb2-29813055-txszf"]
Sep 07 12:15:00.462532 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:15:00.462506    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93449ba3_bf43_4ef1_ace4_04d6c150473f.slice/crio-a260fb43906cecdb7c6f9fcb68603d3636eb835ae009d6baa155c6e9e2165eda WatchSource:0}: Error finding container a260fb43906cecdb7c6f9fcb68603d3636eb835ae009d6baa155c6e9e2165eda: Status 404 returned error can't find the container with id a260fb43906cecdb7c6f9fcb68603d3636eb835ae009d6baa155c6e9e2165eda
Sep 07 12:15:00.466088 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.466068    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bff0c9-29813055-tvgdv"]
Sep 07 12:15:00.467978 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:15:00.467952    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07049ef8_f9a9_42a5_a5fd_56c5bdbad72c.slice/crio-fda649b3053364a8bd33072a464d6a9cfc1ba44dff551a93bd8cb2d25dbcbda5 WatchSource:0}: Error finding container fda649b3053364a8bd33072a464d6a9cfc1ba44dff551a93bd8cb2d25dbcbda5: Status 404 returned error can't find the container with id fda649b3053364a8bd33072a464d6a9cfc1ba44dff551a93bd8cb2d25dbcbda5
Sep 07 12:15:00.531062 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.531031    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-613886-29813055-8pk62"]
Sep 07 12:15:00.533601 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:15:00.533574    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb7c7e70_3e73_415b_b96b_05ca396b92fc.slice/crio-c37c0c85836c7e6f71e21261a2c9e19cf02958a827d3ec61601be10a392a46d3 WatchSource:0}: Error finding container c37c0c85836c7e6f71e21261a2c9e19cf02958a827d3ec61601be10a392a46d3: Status 404 returned error can't find the container with id c37c0c85836c7e6f71e21261a2c9e19cf02958a827d3ec61601be10a392a46d3
Sep 07 12:15:00.551028 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.550774    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-9e422b-29813055-7stbn"]
Sep 07 12:15:00.555171 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:15:00.555032    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e3266a_cffc_4b08_a95c_529f7f81276a.slice/crio-37599afade7931bef0ffa9eadff86993aaef78950e6e66508e0074ad9e8c513f WatchSource:0}: Error finding container 37599afade7931bef0ffa9eadff86993aaef78950e6e66508e0074ad9e8c513f: Status 404 returned error can't find the container with id 37599afade7931bef0ffa9eadff86993aaef78950e6e66508e0074ad9e8c513f
Sep 07 12:15:00.571443 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:00.571417    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f7f063-29813055-lf4kl"]
Sep 07 12:15:00.573923 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:15:00.573895    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a5fcf5d_7fda_4c8d_a306_32b5a35654ff.slice/crio-8dad9fab7ac3f73ad9dec30d6980fd82387eea0075b80e423fbaea09aa5cbc43 WatchSource:0}: Error finding container 8dad9fab7ac3f73ad9dec30d6980fd82387eea0075b80e423fbaea09aa5cbc43: Status 404 returned error can't find the container with id 8dad9fab7ac3f73ad9dec30d6980fd82387eea0075b80e423fbaea09aa5cbc43
Sep 07 12:15:01.470495 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:01.470459    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f7f063-29813055-lf4kl" event={"ID":"4a5fcf5d-7fda-4c8d-a306-32b5a35654ff","Type":"ContainerStarted","Data":"8ad6ca3e42daadcddd410860f8fb358d4b69e61aa7a75ea7c7347b27a7ad5fc4"}
Sep 07 12:15:01.470875 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:01.470527    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f7f063-29813055-lf4kl" event={"ID":"4a5fcf5d-7fda-4c8d-a306-32b5a35654ff","Type":"ContainerStarted","Data":"8dad9fab7ac3f73ad9dec30d6980fd82387eea0075b80e423fbaea09aa5cbc43"}
Sep 07 12:15:01.472242 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:01.472184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bff0c9-29813055-tvgdv" event={"ID":"07049ef8-f9a9-42a5-a5fd-56c5bdbad72c","Type":"ContainerStarted","Data":"4d839e1c0d9a6bccde7212a5acae83de467e0eb935641faf38022d11a2be87de"}
Sep 07 12:15:01.472346 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:01.472269    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bff0c9-29813055-tvgdv" event={"ID":"07049ef8-f9a9-42a5-a5fd-56c5bdbad72c","Type":"ContainerStarted","Data":"fda649b3053364a8bd33072a464d6a9cfc1ba44dff551a93bd8cb2d25dbcbda5"}
Sep 07 12:15:01.473610 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:01.473585    2580 generic.go:356] "Generic (PLEG): container finished" podID="93449ba3-bf43-4ef1-ace4-04d6c150473f" containerID="f8fb8dd4ecdcfdaf32c9e70ac77f2ddcb755d0ed9adad75302f689839c18ced4" exitCode=0
Sep 07 12:15:01.473738 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:01.473615    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3a0bb2-29813055-txszf" event={"ID":"93449ba3-bf43-4ef1-ace4-04d6c150473f","Type":"ContainerDied","Data":"f8fb8dd4ecdcfdaf32c9e70ac77f2ddcb755d0ed9adad75302f689839c18ced4"}
Sep 07 12:15:01.473738 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:01.473664    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3a0bb2-29813055-txszf" event={"ID":"93449ba3-bf43-4ef1-ace4-04d6c150473f","Type":"ContainerStarted","Data":"a260fb43906cecdb7c6f9fcb68603d3636eb835ae009d6baa155c6e9e2165eda"}
Sep 07 12:15:01.475183 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:01.475164    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-9e422b-29813055-7stbn" event={"ID":"65e3266a-cffc-4b08-a95c-529f7f81276a","Type":"ContainerStarted","Data":"c7f54760132a55a92c6cd62f6c6a1f94f5a6be850fe273ecdede70ba86557f6e"}
Sep 07 12:15:01.475280 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:01.475215    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-9e422b-29813055-7stbn" event={"ID":"65e3266a-cffc-4b08-a95c-529f7f81276a","Type":"ContainerStarted","Data":"37599afade7931bef0ffa9eadff86993aaef78950e6e66508e0074ad9e8c513f"}
Sep 07 12:15:01.476698 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:01.476680    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-613886-29813055-8pk62" event={"ID":"bb7c7e70-3e73-415b-b96b-05ca396b92fc","Type":"ContainerStarted","Data":"d824fc593c0c2feb20022fc558f827a611f9e2a70330a0cc11188d7d70e09f4f"}
Sep 07 12:15:01.476769 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:01.476714    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-613886-29813055-8pk62" event={"ID":"bb7c7e70-3e73-415b-b96b-05ca396b92fc","Type":"ContainerStarted","Data":"c37c0c85836c7e6f71e21261a2c9e19cf02958a827d3ec61601be10a392a46d3"}
Sep 07 12:15:01.484924 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:01.484752    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f7f063-29813055-lf4kl" podStartSLOduration=1.484716084 podStartE2EDuration="1.484716084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:15:01.484444392 +0000 UTC m=+1655.980333819" watchObservedRunningTime="2026-09-07 12:15:01.484716084 +0000 UTC m=+1655.980605561"
Sep 07 12:15:01.498922 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:01.498876    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-613886-29813055-8pk62" podStartSLOduration=1.498861634 podStartE2EDuration="1.498861634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:15:01.497786179 +0000 UTC m=+1655.993675620" watchObservedRunningTime="2026-09-07 12:15:01.498861634 +0000 UTC m=+1655.994751050"
Sep 07 12:15:01.513123 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:01.513073    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-9e422b-29813055-7stbn" podStartSLOduration=1.5130576310000001 podStartE2EDuration="1.513057631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:15:01.511225046 +0000 UTC m=+1656.007114473" watchObservedRunningTime="2026-09-07 12:15:01.513057631 +0000 UTC m=+1656.008947106"
Sep 07 12:15:01.538564 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:01.538505    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bff0c9-29813055-tvgdv" podStartSLOduration=1.538485962 podStartE2EDuration="1.538485962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:15:01.536668894 +0000 UTC m=+1656.032558319" watchObservedRunningTime="2026-09-07 12:15:01.538485962 +0000 UTC m=+1656.034375384"
Sep 07 12:15:02.483013 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:02.482978    2580 generic.go:356] "Generic (PLEG): container finished" podID="65e3266a-cffc-4b08-a95c-529f7f81276a" containerID="c7f54760132a55a92c6cd62f6c6a1f94f5a6be850fe273ecdede70ba86557f6e" exitCode=0
Sep 07 12:15:02.483480 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:02.483048    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-9e422b-29813055-7stbn" event={"ID":"65e3266a-cffc-4b08-a95c-529f7f81276a","Type":"ContainerDied","Data":"c7f54760132a55a92c6cd62f6c6a1f94f5a6be850fe273ecdede70ba86557f6e"}
Sep 07 12:15:02.484541 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:02.484514    2580 generic.go:356] "Generic (PLEG): container finished" podID="bb7c7e70-3e73-415b-b96b-05ca396b92fc" containerID="d824fc593c0c2feb20022fc558f827a611f9e2a70330a0cc11188d7d70e09f4f" exitCode=0
Sep 07 12:15:02.484660 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:02.484592    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-613886-29813055-8pk62" event={"ID":"bb7c7e70-3e73-415b-b96b-05ca396b92fc","Type":"ContainerDied","Data":"d824fc593c0c2feb20022fc558f827a611f9e2a70330a0cc11188d7d70e09f4f"}
Sep 07 12:15:02.486000 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:02.485980    2580 generic.go:356] "Generic (PLEG): container finished" podID="4a5fcf5d-7fda-4c8d-a306-32b5a35654ff" containerID="8ad6ca3e42daadcddd410860f8fb358d4b69e61aa7a75ea7c7347b27a7ad5fc4" exitCode=0
Sep 07 12:15:02.486111 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:02.486036    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f7f063-29813055-lf4kl" event={"ID":"4a5fcf5d-7fda-4c8d-a306-32b5a35654ff","Type":"ContainerDied","Data":"8ad6ca3e42daadcddd410860f8fb358d4b69e61aa7a75ea7c7347b27a7ad5fc4"}
Sep 07 12:15:02.487493 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:02.487469    2580 generic.go:356] "Generic (PLEG): container finished" podID="07049ef8-f9a9-42a5-a5fd-56c5bdbad72c" containerID="4d839e1c0d9a6bccde7212a5acae83de467e0eb935641faf38022d11a2be87de" exitCode=0
Sep 07 12:15:02.487585 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:02.487547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bff0c9-29813055-tvgdv" event={"ID":"07049ef8-f9a9-42a5-a5fd-56c5bdbad72c","Type":"ContainerDied","Data":"4d839e1c0d9a6bccde7212a5acae83de467e0eb935641faf38022d11a2be87de"}
Sep 07 12:15:02.671762 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:02.671724    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3a0bb2-29813055-txszf"
Sep 07 12:15:03.494420 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:03.494390    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3a0bb2-29813055-txszf"
Sep 07 12:15:03.494420 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:03.494397    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3a0bb2-29813055-txszf" event={"ID":"93449ba3-bf43-4ef1-ace4-04d6c150473f","Type":"ContainerDied","Data":"a260fb43906cecdb7c6f9fcb68603d3636eb835ae009d6baa155c6e9e2165eda"}
Sep 07 12:15:03.494935 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:03.494454    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a260fb43906cecdb7c6f9fcb68603d3636eb835ae009d6baa155c6e9e2165eda"
Sep 07 12:15:03.656119 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:03.656089    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-613886-29813055-8pk62"
Sep 07 12:15:03.694051 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:03.694023    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bff0c9-29813055-tvgdv"
Sep 07 12:15:03.726059 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:03.726027    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f7f063-29813055-lf4kl"
Sep 07 12:15:03.730568 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:03.730544    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-9e422b-29813055-7stbn"
Sep 07 12:15:04.501383 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:04.501356    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-9e422b-29813055-7stbn"
Sep 07 12:15:04.501821 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:04.501357    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-9e422b-29813055-7stbn" event={"ID":"65e3266a-cffc-4b08-a95c-529f7f81276a","Type":"ContainerDied","Data":"37599afade7931bef0ffa9eadff86993aaef78950e6e66508e0074ad9e8c513f"}
Sep 07 12:15:04.501821 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:04.501491    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37599afade7931bef0ffa9eadff86993aaef78950e6e66508e0074ad9e8c513f"
Sep 07 12:15:04.502759 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:04.502737    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-613886-29813055-8pk62"
Sep 07 12:15:04.502759 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:04.502753    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-613886-29813055-8pk62" event={"ID":"bb7c7e70-3e73-415b-b96b-05ca396b92fc","Type":"ContainerDied","Data":"c37c0c85836c7e6f71e21261a2c9e19cf02958a827d3ec61601be10a392a46d3"}
Sep 07 12:15:04.502931 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:04.502794    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c37c0c85836c7e6f71e21261a2c9e19cf02958a827d3ec61601be10a392a46d3"
Sep 07 12:15:04.504093 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:04.504075    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f7f063-29813055-lf4kl"
Sep 07 12:15:04.504093 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:04.504085    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f7f063-29813055-lf4kl" event={"ID":"4a5fcf5d-7fda-4c8d-a306-32b5a35654ff","Type":"ContainerDied","Data":"8dad9fab7ac3f73ad9dec30d6980fd82387eea0075b80e423fbaea09aa5cbc43"}
Sep 07 12:15:04.504243 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:04.504123    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8dad9fab7ac3f73ad9dec30d6980fd82387eea0075b80e423fbaea09aa5cbc43"
Sep 07 12:15:04.505575 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:04.505555    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bff0c9-29813055-tvgdv" event={"ID":"07049ef8-f9a9-42a5-a5fd-56c5bdbad72c","Type":"ContainerDied","Data":"fda649b3053364a8bd33072a464d6a9cfc1ba44dff551a93bd8cb2d25dbcbda5"}
Sep 07 12:15:04.505663 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:04.505587    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fda649b3053364a8bd33072a464d6a9cfc1ba44dff551a93bd8cb2d25dbcbda5"
Sep 07 12:15:04.505663 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:04.505559    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bff0c9-29813055-tvgdv"
Sep 07 12:15:08.173882 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.173840    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr"]
Sep 07 12:15:08.174263 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.174238    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65e3266a-cffc-4b08-a95c-529f7f81276a" containerName="cleanup"
Sep 07 12:15:08.174263 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.174250    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="65e3266a-cffc-4b08-a95c-529f7f81276a" containerName="cleanup"
Sep 07 12:15:08.174263 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.174260    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb7c7e70-3e73-415b-b96b-05ca396b92fc" containerName="cleanup"
Sep 07 12:15:08.174263 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.174265    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb7c7e70-3e73-415b-b96b-05ca396b92fc" containerName="cleanup"
Sep 07 12:15:08.174411 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.174281    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a5fcf5d-7fda-4c8d-a306-32b5a35654ff" containerName="cleanup"
Sep 07 12:15:08.174411 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.174287    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a5fcf5d-7fda-4c8d-a306-32b5a35654ff" containerName="cleanup"
Sep 07 12:15:08.174411 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.174300    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07049ef8-f9a9-42a5-a5fd-56c5bdbad72c" containerName="cleanup"
Sep 07 12:15:08.174411 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.174305    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="07049ef8-f9a9-42a5-a5fd-56c5bdbad72c" containerName="cleanup"
Sep 07 12:15:08.174411 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.174318    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93449ba3-bf43-4ef1-ace4-04d6c150473f" containerName="cleanup"
Sep 07 12:15:08.174411 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.174323    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="93449ba3-bf43-4ef1-ace4-04d6c150473f" containerName="cleanup"
Sep 07 12:15:08.174411 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.174380    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="65e3266a-cffc-4b08-a95c-529f7f81276a" containerName="cleanup"
Sep 07 12:15:08.174411 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.174387    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="07049ef8-f9a9-42a5-a5fd-56c5bdbad72c" containerName="cleanup"
Sep 07 12:15:08.174411 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.174393    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a5fcf5d-7fda-4c8d-a306-32b5a35654ff" containerName="cleanup"
Sep 07 12:15:08.174411 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.174401    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb7c7e70-3e73-415b-b96b-05ca396b92fc" containerName="cleanup"
Sep 07 12:15:08.174411 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.174410    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="93449ba3-bf43-4ef1-ace4-04d6c150473f" containerName="cleanup"
Sep 07 12:15:08.177169 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.177152    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr"
Sep 07 12:15:08.179533 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.179508    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:15:08.179533 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.179518    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:15:08.183648 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.183614    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr"]
Sep 07 12:15:08.273218 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.273169    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e8abc1f-3bcb-40f3-a5cf-950b0145d47c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr\" (UID: \"5e8abc1f-3bcb-40f3-a5cf-950b0145d47c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr"
Sep 07 12:15:08.374408 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.374370    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e8abc1f-3bcb-40f3-a5cf-950b0145d47c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr\" (UID: \"5e8abc1f-3bcb-40f3-a5cf-950b0145d47c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr"
Sep 07 12:15:08.375008 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.374986    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e8abc1f-3bcb-40f3-a5cf-950b0145d47c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr\" (UID: \"5e8abc1f-3bcb-40f3-a5cf-950b0145d47c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr"
Sep 07 12:15:08.489366 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.489280    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr"
Sep 07 12:15:08.760256 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:08.760229    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr"]
Sep 07 12:15:08.761856 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:15:08.761831    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e8abc1f_3bcb_40f3_a5cf_950b0145d47c.slice/crio-487259b37c301063c0ef42164aaa2c90023cd2fd707229f2a45b976448677a8d WatchSource:0}: Error finding container 487259b37c301063c0ef42164aaa2c90023cd2fd707229f2a45b976448677a8d: Status 404 returned error can't find the container with id 487259b37c301063c0ef42164aaa2c90023cd2fd707229f2a45b976448677a8d
Sep 07 12:15:09.539382 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:09.539347    2580 generic.go:356] "Generic (PLEG): container finished" podID="5e8abc1f-3bcb-40f3-a5cf-950b0145d47c" containerID="0cee2927890ccd097010f3aba4eb7fb0ca34acfababfda0785182b8a6cb5850e" exitCode=0
Sep 07 12:15:09.539382 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:09.539387    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr" event={"ID":"5e8abc1f-3bcb-40f3-a5cf-950b0145d47c","Type":"ContainerDied","Data":"0cee2927890ccd097010f3aba4eb7fb0ca34acfababfda0785182b8a6cb5850e"}
Sep 07 12:15:09.539803 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:09.539429    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr" event={"ID":"5e8abc1f-3bcb-40f3-a5cf-950b0145d47c","Type":"ContainerStarted","Data":"487259b37c301063c0ef42164aaa2c90023cd2fd707229f2a45b976448677a8d"}
Sep 07 12:15:10.681090 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:10.681059    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr"
Sep 07 12:15:10.798319 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:10.798219    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e8abc1f-3bcb-40f3-a5cf-950b0145d47c-maas-api-service-ca\") pod \"5e8abc1f-3bcb-40f3-a5cf-950b0145d47c\" (UID: \"5e8abc1f-3bcb-40f3-a5cf-950b0145d47c\") "
Sep 07 12:15:10.798550 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:10.798527    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e8abc1f-3bcb-40f3-a5cf-950b0145d47c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5e8abc1f-3bcb-40f3-a5cf-950b0145d47c" (UID: "5e8abc1f-3bcb-40f3-a5cf-950b0145d47c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:15:10.899642 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:10.899602    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e8abc1f-3bcb-40f3-a5cf-950b0145d47c-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:15:11.556759 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:11.556734    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr"
Sep 07 12:15:11.556759 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:11.556746    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr" event={"ID":"5e8abc1f-3bcb-40f3-a5cf-950b0145d47c","Type":"ContainerDied","Data":"487259b37c301063c0ef42164aaa2c90023cd2fd707229f2a45b976448677a8d"}
Sep 07 12:15:11.556970 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:11.556818    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="487259b37c301063c0ef42164aaa2c90023cd2fd707229f2a45b976448677a8d"
Sep 07 12:15:23.710755 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:23.710714    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f7f063-29813055-lf4kl"]
Sep 07 12:15:23.713005 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:23.712976    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f7f063-29813055-lf4kl"]
Sep 07 12:15:23.733743 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:23.733690    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"]
Sep 07 12:15:23.734125 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:23.733952    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv" podUID="89c1084a-15ca-456b-a8e5-5b0eedd872eb" containerName="payload-processing" containerID="cri-o://d5a6ee99def65568871ba52a7b636b8bdcdaf7a20825d6a28e5ecc979e3eb360" gracePeriod=30
Sep 07 12:15:23.742635 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:23.742604    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"]
Sep 07 12:15:23.742911 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:23.742872    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck" podUID="a6b1e0f2-583a-4a70-af8a-337ec354424f" containerName="payload-pre-processing" containerID="cri-o://ca3245d3dbe29a67d5e4dc5b5456ffa1f324afdaac27c1747f206f6a1042b343" gracePeriod=30
Sep 07 12:15:23.913299 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:23.912952    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr"]
Sep 07 12:15:23.915788 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:23.915760    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f7f063-pqtjr"]
Sep 07 12:15:24.012046 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.012017    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a5fcf5d-7fda-4c8d-a306-32b5a35654ff" path="/var/lib/kubelet/pods/4a5fcf5d-7fda-4c8d-a306-32b5a35654ff/volumes"
Sep 07 12:15:24.012373 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.012358    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e8abc1f-3bcb-40f3-a5cf-950b0145d47c" path="/var/lib/kubelet/pods/5e8abc1f-3bcb-40f3-a5cf-950b0145d47c/volumes"
Sep 07 12:15:24.024717 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.024696    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"
Sep 07 12:15:24.026902 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.026884    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"
Sep 07 12:15:24.115990 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.115946    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ld82m\" (UniqueName: \"kubernetes.io/projected/89c1084a-15ca-456b-a8e5-5b0eedd872eb-kube-api-access-ld82m\") pod \"89c1084a-15ca-456b-a8e5-5b0eedd872eb\" (UID: \"89c1084a-15ca-456b-a8e5-5b0eedd872eb\") "
Sep 07 12:15:24.116226 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.116055    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6b1e0f2-583a-4a70-af8a-337ec354424f-plugins-config-volume\") pod \"a6b1e0f2-583a-4a70-af8a-337ec354424f\" (UID: \"a6b1e0f2-583a-4a70-af8a-337ec354424f\") "
Sep 07 12:15:24.116226 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.116167    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89c1084a-15ca-456b-a8e5-5b0eedd872eb-plugins-config-volume\") pod \"89c1084a-15ca-456b-a8e5-5b0eedd872eb\" (UID: \"89c1084a-15ca-456b-a8e5-5b0eedd872eb\") "
Sep 07 12:15:24.116500 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.116354    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trk2r\" (UniqueName: \"kubernetes.io/projected/a6b1e0f2-583a-4a70-af8a-337ec354424f-kube-api-access-trk2r\") pod \"a6b1e0f2-583a-4a70-af8a-337ec354424f\" (UID: \"a6b1e0f2-583a-4a70-af8a-337ec354424f\") "
Sep 07 12:15:24.116500 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.116466    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6b1e0f2-583a-4a70-af8a-337ec354424f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a6b1e0f2-583a-4a70-af8a-337ec354424f" (UID: "a6b1e0f2-583a-4a70-af8a-337ec354424f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:15:24.116668 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.116574    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89c1084a-15ca-456b-a8e5-5b0eedd872eb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "89c1084a-15ca-456b-a8e5-5b0eedd872eb" (UID: "89c1084a-15ca-456b-a8e5-5b0eedd872eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:15:24.116761 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.116748    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6b1e0f2-583a-4a70-af8a-337ec354424f-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:15:24.116833 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.116766    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89c1084a-15ca-456b-a8e5-5b0eedd872eb-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:15:24.118709 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.118682    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89c1084a-15ca-456b-a8e5-5b0eedd872eb-kube-api-access-ld82m" (OuterVolumeSpecName: "kube-api-access-ld82m") pod "89c1084a-15ca-456b-a8e5-5b0eedd872eb" (UID: "89c1084a-15ca-456b-a8e5-5b0eedd872eb"). InnerVolumeSpecName "kube-api-access-ld82m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:15:24.118831 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.118803    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6b1e0f2-583a-4a70-af8a-337ec354424f-kube-api-access-trk2r" (OuterVolumeSpecName: "kube-api-access-trk2r") pod "a6b1e0f2-583a-4a70-af8a-337ec354424f" (UID: "a6b1e0f2-583a-4a70-af8a-337ec354424f"). InnerVolumeSpecName "kube-api-access-trk2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:15:24.217883 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.217838    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trk2r\" (UniqueName: \"kubernetes.io/projected/a6b1e0f2-583a-4a70-af8a-337ec354424f-kube-api-access-trk2r\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:15:24.217883 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.217876    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld82m\" (UniqueName: \"kubernetes.io/projected/89c1084a-15ca-456b-a8e5-5b0eedd872eb-kube-api-access-ld82m\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:15:24.629608 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.629573    2580 generic.go:356] "Generic (PLEG): container finished" podID="a6b1e0f2-583a-4a70-af8a-337ec354424f" containerID="ca3245d3dbe29a67d5e4dc5b5456ffa1f324afdaac27c1747f206f6a1042b343" exitCode=0
Sep 07 12:15:24.629827 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.629617    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"
Sep 07 12:15:24.629827 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.629667    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck" event={"ID":"a6b1e0f2-583a-4a70-af8a-337ec354424f","Type":"ContainerDied","Data":"ca3245d3dbe29a67d5e4dc5b5456ffa1f324afdaac27c1747f206f6a1042b343"}
Sep 07 12:15:24.629827 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.629730    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck" event={"ID":"a6b1e0f2-583a-4a70-af8a-337ec354424f","Type":"ContainerDied","Data":"03e5dda375a7fb4e77a4ddbe059962e8baa601003f785c77cc9044590e5826d4"}
Sep 07 12:15:24.629827 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.629764    2580 scope.go:117] "RemoveContainer" containerID="ca3245d3dbe29a67d5e4dc5b5456ffa1f324afdaac27c1747f206f6a1042b343"
Sep 07 12:15:24.631232 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.631191    2580 generic.go:356] "Generic (PLEG): container finished" podID="89c1084a-15ca-456b-a8e5-5b0eedd872eb" containerID="d5a6ee99def65568871ba52a7b636b8bdcdaf7a20825d6a28e5ecc979e3eb360" exitCode=0
Sep 07 12:15:24.631342 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.631234    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv" event={"ID":"89c1084a-15ca-456b-a8e5-5b0eedd872eb","Type":"ContainerDied","Data":"d5a6ee99def65568871ba52a7b636b8bdcdaf7a20825d6a28e5ecc979e3eb360"}
Sep 07 12:15:24.631342 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.631256    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"
Sep 07 12:15:24.631342 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.631279    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv" event={"ID":"89c1084a-15ca-456b-a8e5-5b0eedd872eb","Type":"ContainerDied","Data":"db014052dabe28f21aa426cce3c37a8351676353fb47cdbe6f5d269eab81789f"}
Sep 07 12:15:24.639592 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.639525    2580 scope.go:117] "RemoveContainer" containerID="ca3245d3dbe29a67d5e4dc5b5456ffa1f324afdaac27c1747f206f6a1042b343"
Sep 07 12:15:24.639967 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:15:24.639946    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca3245d3dbe29a67d5e4dc5b5456ffa1f324afdaac27c1747f206f6a1042b343\": container with ID starting with ca3245d3dbe29a67d5e4dc5b5456ffa1f324afdaac27c1747f206f6a1042b343 not found: ID does not exist" containerID="ca3245d3dbe29a67d5e4dc5b5456ffa1f324afdaac27c1747f206f6a1042b343"
Sep 07 12:15:24.640068 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.639979    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca3245d3dbe29a67d5e4dc5b5456ffa1f324afdaac27c1747f206f6a1042b343"} err="failed to get container status \"ca3245d3dbe29a67d5e4dc5b5456ffa1f324afdaac27c1747f206f6a1042b343\": rpc error: code = NotFound desc = could not find container \"ca3245d3dbe29a67d5e4dc5b5456ffa1f324afdaac27c1747f206f6a1042b343\": container with ID starting with ca3245d3dbe29a67d5e4dc5b5456ffa1f324afdaac27c1747f206f6a1042b343 not found: ID does not exist"
Sep 07 12:15:24.640068 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.640004    2580 scope.go:117] "RemoveContainer" containerID="d5a6ee99def65568871ba52a7b636b8bdcdaf7a20825d6a28e5ecc979e3eb360"
Sep 07 12:15:24.648570 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.648546    2580 scope.go:117] "RemoveContainer" containerID="d5a6ee99def65568871ba52a7b636b8bdcdaf7a20825d6a28e5ecc979e3eb360"
Sep 07 12:15:24.648884 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:15:24.648865    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5a6ee99def65568871ba52a7b636b8bdcdaf7a20825d6a28e5ecc979e3eb360\": container with ID starting with d5a6ee99def65568871ba52a7b636b8bdcdaf7a20825d6a28e5ecc979e3eb360 not found: ID does not exist" containerID="d5a6ee99def65568871ba52a7b636b8bdcdaf7a20825d6a28e5ecc979e3eb360"
Sep 07 12:15:24.648947 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.648892    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5a6ee99def65568871ba52a7b636b8bdcdaf7a20825d6a28e5ecc979e3eb360"} err="failed to get container status \"d5a6ee99def65568871ba52a7b636b8bdcdaf7a20825d6a28e5ecc979e3eb360\": rpc error: code = NotFound desc = could not find container \"d5a6ee99def65568871ba52a7b636b8bdcdaf7a20825d6a28e5ecc979e3eb360\": container with ID starting with d5a6ee99def65568871ba52a7b636b8bdcdaf7a20825d6a28e5ecc979e3eb360 not found: ID does not exist"
Sep 07 12:15:24.652489 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.652460    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"]
Sep 07 12:15:24.656837 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.656810    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f7f063-9764d7bdb-rhxck"]
Sep 07 12:15:24.666960 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.666927    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"]
Sep 07 12:15:24.670856 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:24.670827    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f7f063-66d477497f-9bdpv"]
Sep 07 12:15:26.010946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:26.010912    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89c1084a-15ca-456b-a8e5-5b0eedd872eb" path="/var/lib/kubelet/pods/89c1084a-15ca-456b-a8e5-5b0eedd872eb/volumes"
Sep 07 12:15:26.011351 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:26.011305    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6b1e0f2-583a-4a70-af8a-337ec354424f" path="/var/lib/kubelet/pods/a6b1e0f2-583a-4a70-af8a-337ec354424f/volumes"
Sep 07 12:15:26.228987 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:26.228952    2580 scope.go:117] "RemoveContainer" containerID="692ec19083f0c1c1bb94b60887432fd80d081244ad728ee4b29f8b6505d66a42"
Sep 07 12:15:26.242170 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:26.242150    2580 scope.go:117] "RemoveContainer" containerID="209ec40e60657148539d1a07c784f62f8a52f7a2a6289e83f6223d9561d9305a"
Sep 07 12:15:26.250353 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:26.250325    2580 scope.go:117] "RemoveContainer" containerID="1ff464a6a6b750d1fab3d1a2ef12623ae4b3f80c493f6b62a3a33353f7cbd3cd"
Sep 07 12:15:30.899809 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:30.899768    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9"]
Sep 07 12:15:30.900266 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:30.900220    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89c1084a-15ca-456b-a8e5-5b0eedd872eb" containerName="payload-processing"
Sep 07 12:15:30.900319 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:30.900270    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="89c1084a-15ca-456b-a8e5-5b0eedd872eb" containerName="payload-processing"
Sep 07 12:15:30.900319 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:30.900296    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6b1e0f2-583a-4a70-af8a-337ec354424f" containerName="payload-pre-processing"
Sep 07 12:15:30.900319 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:30.900305    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6b1e0f2-583a-4a70-af8a-337ec354424f" containerName="payload-pre-processing"
Sep 07 12:15:30.900319 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:30.900316    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e8abc1f-3bcb-40f3-a5cf-950b0145d47c" containerName="revoke-keys"
Sep 07 12:15:30.900446 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:30.900324    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e8abc1f-3bcb-40f3-a5cf-950b0145d47c" containerName="revoke-keys"
Sep 07 12:15:30.900446 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:30.900407    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e8abc1f-3bcb-40f3-a5cf-950b0145d47c" containerName="revoke-keys"
Sep 07 12:15:30.900446 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:30.900418    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="89c1084a-15ca-456b-a8e5-5b0eedd872eb" containerName="payload-processing"
Sep 07 12:15:30.900446 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:30.900424    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6b1e0f2-583a-4a70-af8a-337ec354424f" containerName="payload-pre-processing"
Sep 07 12:15:30.903417 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:30.903398    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9"
Sep 07 12:15:30.905945 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:30.905921    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:15:30.906050 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:30.905930    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:15:30.910032 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:30.910005    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9"]
Sep 07 12:15:30.986492 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:30.986446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d85ddc8-b09a-491b-b094-d93055a77831-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9\" (UID: \"2d85ddc8-b09a-491b-b094-d93055a77831\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9"
Sep 07 12:15:31.087491 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:31.087451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d85ddc8-b09a-491b-b094-d93055a77831-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9\" (UID: \"2d85ddc8-b09a-491b-b094-d93055a77831\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9"
Sep 07 12:15:31.088078 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:31.088059    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d85ddc8-b09a-491b-b094-d93055a77831-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9\" (UID: \"2d85ddc8-b09a-491b-b094-d93055a77831\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9"
Sep 07 12:15:31.215375 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:31.215272    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9"
Sep 07 12:15:31.496007 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:31.488942    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9"]
Sep 07 12:15:31.672637 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:31.672600    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9" event={"ID":"2d85ddc8-b09a-491b-b094-d93055a77831","Type":"ContainerStarted","Data":"08f298d6086f01ecfbb4f3d11baef4095fa7ca2beef5734d31b71d471b400267"}
Sep 07 12:15:32.680035 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:32.680003    2580 generic.go:356] "Generic (PLEG): container finished" podID="2d85ddc8-b09a-491b-b094-d93055a77831" containerID="ba3c4a2922ca7c7e9a2a55fc12bf4e81a49aa5daac1bfefde84d9202ae066431" exitCode=0
Sep 07 12:15:32.680421 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:32.680082    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9" event={"ID":"2d85ddc8-b09a-491b-b094-d93055a77831","Type":"ContainerDied","Data":"ba3c4a2922ca7c7e9a2a55fc12bf4e81a49aa5daac1bfefde84d9202ae066431"}
Sep 07 12:15:33.818532 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:33.818506    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9"
Sep 07 12:15:33.916251 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:33.916161    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d85ddc8-b09a-491b-b094-d93055a77831-maas-api-service-ca\") pod \"2d85ddc8-b09a-491b-b094-d93055a77831\" (UID: \"2d85ddc8-b09a-491b-b094-d93055a77831\") "
Sep 07 12:15:33.916551 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:33.916529    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d85ddc8-b09a-491b-b094-d93055a77831-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2d85ddc8-b09a-491b-b094-d93055a77831" (UID: "2d85ddc8-b09a-491b-b094-d93055a77831"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:15:34.017179 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:34.017086    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d85ddc8-b09a-491b-b094-d93055a77831-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:15:34.691052 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:34.691020    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9"
Sep 07 12:15:34.691264 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:34.691050    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9" event={"ID":"2d85ddc8-b09a-491b-b094-d93055a77831","Type":"ContainerDied","Data":"08f298d6086f01ecfbb4f3d11baef4095fa7ca2beef5734d31b71d471b400267"}
Sep 07 12:15:34.691264 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:34.691097    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08f298d6086f01ecfbb4f3d11baef4095fa7ca2beef5734d31b71d471b400267"
Sep 07 12:15:46.442864 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.442830    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-9e422b-29813055-7stbn"]
Sep 07 12:15:46.446138 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.446104    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-9e422b-29813055-7stbn"]
Sep 07 12:15:46.474131 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.474098    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"]
Sep 07 12:15:46.474475 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.474435    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx" podUID="07aa3fa4-debd-4518-9d50-da879148cee8" containerName="payload-pre-processing" containerID="cri-o://ab31d77ccf8aecdb86aa74e9e258e47aba955c723cb154aedfc43fa2c132a069" gracePeriod=30
Sep 07 12:15:46.638797 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.638760    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9"]
Sep 07 12:15:46.642073 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.642042    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9e422b-qt2c9"]
Sep 07 12:15:46.753791 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.753652    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"
Sep 07 12:15:46.755927 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.755894    2580 generic.go:356] "Generic (PLEG): container finished" podID="07aa3fa4-debd-4518-9d50-da879148cee8" containerID="ab31d77ccf8aecdb86aa74e9e258e47aba955c723cb154aedfc43fa2c132a069" exitCode=0
Sep 07 12:15:46.756094 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.755939    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"
Sep 07 12:15:46.756094 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.755970    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx" event={"ID":"07aa3fa4-debd-4518-9d50-da879148cee8","Type":"ContainerDied","Data":"ab31d77ccf8aecdb86aa74e9e258e47aba955c723cb154aedfc43fa2c132a069"}
Sep 07 12:15:46.756094 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.756022    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx" event={"ID":"07aa3fa4-debd-4518-9d50-da879148cee8","Type":"ContainerDied","Data":"c09b5af3f712f2a2b9c72eaa729b0e34ae8f120e0d89a164dec8519f51b304e0"}
Sep 07 12:15:46.756094 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.756047    2580 scope.go:117] "RemoveContainer" containerID="ab31d77ccf8aecdb86aa74e9e258e47aba955c723cb154aedfc43fa2c132a069"
Sep 07 12:15:46.765550 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.765524    2580 scope.go:117] "RemoveContainer" containerID="ab31d77ccf8aecdb86aa74e9e258e47aba955c723cb154aedfc43fa2c132a069"
Sep 07 12:15:46.765893 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:15:46.765874    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab31d77ccf8aecdb86aa74e9e258e47aba955c723cb154aedfc43fa2c132a069\": container with ID starting with ab31d77ccf8aecdb86aa74e9e258e47aba955c723cb154aedfc43fa2c132a069 not found: ID does not exist" containerID="ab31d77ccf8aecdb86aa74e9e258e47aba955c723cb154aedfc43fa2c132a069"
Sep 07 12:15:46.765971 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.765902    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab31d77ccf8aecdb86aa74e9e258e47aba955c723cb154aedfc43fa2c132a069"} err="failed to get container status \"ab31d77ccf8aecdb86aa74e9e258e47aba955c723cb154aedfc43fa2c132a069\": rpc error: code = NotFound desc = could not find container \"ab31d77ccf8aecdb86aa74e9e258e47aba955c723cb154aedfc43fa2c132a069\": container with ID starting with ab31d77ccf8aecdb86aa74e9e258e47aba955c723cb154aedfc43fa2c132a069 not found: ID does not exist"
Sep 07 12:15:46.851408 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.851365    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07aa3fa4-debd-4518-9d50-da879148cee8-plugins-config-volume\") pod \"07aa3fa4-debd-4518-9d50-da879148cee8\" (UID: \"07aa3fa4-debd-4518-9d50-da879148cee8\") "
Sep 07 12:15:46.851611 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.851510    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbc7p\" (UniqueName: \"kubernetes.io/projected/07aa3fa4-debd-4518-9d50-da879148cee8-kube-api-access-rbc7p\") pod \"07aa3fa4-debd-4518-9d50-da879148cee8\" (UID: \"07aa3fa4-debd-4518-9d50-da879148cee8\") "
Sep 07 12:15:46.851732 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.851706    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07aa3fa4-debd-4518-9d50-da879148cee8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "07aa3fa4-debd-4518-9d50-da879148cee8" (UID: "07aa3fa4-debd-4518-9d50-da879148cee8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:15:46.851830 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.851816    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07aa3fa4-debd-4518-9d50-da879148cee8-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:15:46.853531 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.853503    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07aa3fa4-debd-4518-9d50-da879148cee8-kube-api-access-rbc7p" (OuterVolumeSpecName: "kube-api-access-rbc7p") pod "07aa3fa4-debd-4518-9d50-da879148cee8" (UID: "07aa3fa4-debd-4518-9d50-da879148cee8"). InnerVolumeSpecName "kube-api-access-rbc7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:15:46.952684 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:46.952641    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbc7p\" (UniqueName: \"kubernetes.io/projected/07aa3fa4-debd-4518-9d50-da879148cee8-kube-api-access-rbc7p\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:15:47.077676 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:47.077638    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"]
Sep 07 12:15:47.081630 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:47.081599    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9e422b-6867bb6d94-q6gcx"]
Sep 07 12:15:48.010650 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:48.010565    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07aa3fa4-debd-4518-9d50-da879148cee8" path="/var/lib/kubelet/pods/07aa3fa4-debd-4518-9d50-da879148cee8/volumes"
Sep 07 12:15:48.011038 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:48.010954    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d85ddc8-b09a-491b-b094-d93055a77831" path="/var/lib/kubelet/pods/2d85ddc8-b09a-491b-b094-d93055a77831/volumes"
Sep 07 12:15:48.011241 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:48.011227    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65e3266a-cffc-4b08-a95c-529f7f81276a" path="/var/lib/kubelet/pods/65e3266a-cffc-4b08-a95c-529f7f81276a/volumes"
Sep 07 12:15:53.287022 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:53.286987    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz"]
Sep 07 12:15:53.287433 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:53.287399    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07aa3fa4-debd-4518-9d50-da879148cee8" containerName="payload-pre-processing"
Sep 07 12:15:53.287433 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:53.287413    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="07aa3fa4-debd-4518-9d50-da879148cee8" containerName="payload-pre-processing"
Sep 07 12:15:53.287433 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:53.287421    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d85ddc8-b09a-491b-b094-d93055a77831" containerName="revoke-keys"
Sep 07 12:15:53.287433 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:53.287427    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d85ddc8-b09a-491b-b094-d93055a77831" containerName="revoke-keys"
Sep 07 12:15:53.287566 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:53.287498    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="07aa3fa4-debd-4518-9d50-da879148cee8" containerName="payload-pre-processing"
Sep 07 12:15:53.287566 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:53.287506    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d85ddc8-b09a-491b-b094-d93055a77831" containerName="revoke-keys"
Sep 07 12:15:53.291946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:53.291927    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz"
Sep 07 12:15:53.295247 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:53.295188    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:15:53.295392 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:53.295291    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:15:53.296869 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:53.296843    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz"]
Sep 07 12:15:53.411507 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:53.411466    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c4758af-ca73-4293-8876-db34df5fc8ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz\" (UID: \"8c4758af-ca73-4293-8876-db34df5fc8ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz"
Sep 07 12:15:53.512424 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:53.512368    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c4758af-ca73-4293-8876-db34df5fc8ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz\" (UID: \"8c4758af-ca73-4293-8876-db34df5fc8ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz"
Sep 07 12:15:53.513078 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:53.513053    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c4758af-ca73-4293-8876-db34df5fc8ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz\" (UID: \"8c4758af-ca73-4293-8876-db34df5fc8ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz"
Sep 07 12:15:53.604673 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:53.604573    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz"
Sep 07 12:15:53.879229 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:53.879168    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz"]
Sep 07 12:15:53.881704 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:15:53.881672    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c4758af_ca73_4293_8876_db34df5fc8ca.slice/crio-ced45a52744d44373a037b647cb2d9eb2d9d2acd6efaf6ef1bb0a2161ab251a8 WatchSource:0}: Error finding container ced45a52744d44373a037b647cb2d9eb2d9d2acd6efaf6ef1bb0a2161ab251a8: Status 404 returned error can't find the container with id ced45a52744d44373a037b647cb2d9eb2d9d2acd6efaf6ef1bb0a2161ab251a8
Sep 07 12:15:54.800056 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:54.800018    2580 generic.go:356] "Generic (PLEG): container finished" podID="8c4758af-ca73-4293-8876-db34df5fc8ca" containerID="533128732d31143730cfabdf295fee156a0e0645328e32cb5c9e207acebf7c91" exitCode=0
Sep 07 12:15:54.800571 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:54.800108    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz" event={"ID":"8c4758af-ca73-4293-8876-db34df5fc8ca","Type":"ContainerDied","Data":"533128732d31143730cfabdf295fee156a0e0645328e32cb5c9e207acebf7c91"}
Sep 07 12:15:54.800571 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:54.800159    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz" event={"ID":"8c4758af-ca73-4293-8876-db34df5fc8ca","Type":"ContainerStarted","Data":"ced45a52744d44373a037b647cb2d9eb2d9d2acd6efaf6ef1bb0a2161ab251a8"}
Sep 07 12:15:55.935638 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:55.935607    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz"
Sep 07 12:15:56.036559 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:56.036520    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c4758af-ca73-4293-8876-db34df5fc8ca-maas-api-service-ca\") pod \"8c4758af-ca73-4293-8876-db34df5fc8ca\" (UID: \"8c4758af-ca73-4293-8876-db34df5fc8ca\") "
Sep 07 12:15:56.036928 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:56.036906    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c4758af-ca73-4293-8876-db34df5fc8ca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8c4758af-ca73-4293-8876-db34df5fc8ca" (UID: "8c4758af-ca73-4293-8876-db34df5fc8ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:15:56.137648 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:56.137612    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c4758af-ca73-4293-8876-db34df5fc8ca-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:15:56.812460 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:56.812421    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz" event={"ID":"8c4758af-ca73-4293-8876-db34df5fc8ca","Type":"ContainerDied","Data":"ced45a52744d44373a037b647cb2d9eb2d9d2acd6efaf6ef1bb0a2161ab251a8"}
Sep 07 12:15:56.812460 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:56.812457    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz"
Sep 07 12:15:56.812721 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:15:56.812498    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ced45a52744d44373a037b647cb2d9eb2d9d2acd6efaf6ef1bb0a2161ab251a8"
Sep 07 12:16:03.916167 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:03.916110    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3a0bb2-29813055-txszf"]
Sep 07 12:16:03.917863 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:03.917832    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3a0bb2-29813055-txszf"]
Sep 07 12:16:04.009897 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:04.009863    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93449ba3-bf43-4ef1-ace4-04d6c150473f" path="/var/lib/kubelet/pods/93449ba3-bf43-4ef1-ace4-04d6c150473f/volumes"
Sep 07 12:16:04.100489 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:04.100454    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz"]
Sep 07 12:16:04.103321 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:04.103290    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3a0bb2-9lllz"]
Sep 07 12:16:06.010615 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:06.010582    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c4758af-ca73-4293-8876-db34df5fc8ca" path="/var/lib/kubelet/pods/8c4758af-ca73-4293-8876-db34df5fc8ca/volumes"
Sep 07 12:16:10.649331 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:10.649291    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"]
Sep 07 12:16:10.649818 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:10.649590    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf" podUID="3077305f-0902-4007-a3e6-beabfbb2a12d" containerName="istio-proxy" containerID="cri-o://e58febf34d8e6a5136fcf0d494f29185fd11f9c5355e3765c9eeb289ecacfe4b" gracePeriod=30
Sep 07 12:16:11.026446 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:11.026348    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq"]
Sep 07 12:16:11.027126 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:11.027105    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c4758af-ca73-4293-8876-db34df5fc8ca" containerName="revoke-keys"
Sep 07 12:16:11.027170 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:11.027136    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c4758af-ca73-4293-8876-db34df5fc8ca" containerName="revoke-keys"
Sep 07 12:16:11.027268 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:11.027257    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c4758af-ca73-4293-8876-db34df5fc8ca" containerName="revoke-keys"
Sep 07 12:16:11.030684 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:11.030664    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq"
Sep 07 12:16:11.034031 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:11.034003    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:16:11.034478 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:11.034449    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:16:11.036269 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:11.036243    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq"]
Sep 07 12:16:11.075730 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:11.075697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d160f8d3-6410-4282-bba6-bd734acbd278-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq\" (UID: \"d160f8d3-6410-4282-bba6-bd734acbd278\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq"
Sep 07 12:16:11.177105 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:11.177060    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d160f8d3-6410-4282-bba6-bd734acbd278-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq\" (UID: \"d160f8d3-6410-4282-bba6-bd734acbd278\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq"
Sep 07 12:16:11.177687 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:11.177665    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d160f8d3-6410-4282-bba6-bd734acbd278-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq\" (UID: \"d160f8d3-6410-4282-bba6-bd734acbd278\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq"
Sep 07 12:16:11.361190 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:11.361102    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq"
Sep 07 12:16:11.631234 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:11.631206    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq"]
Sep 07 12:16:11.633852 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:16:11.633823    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd160f8d3_6410_4282_bba6_bd734acbd278.slice/crio-cf0c55d9dab43276787c5b224136d312c14337535bc6854a1ace408a0c34b482 WatchSource:0}: Error finding container cf0c55d9dab43276787c5b224136d312c14337535bc6854a1ace408a0c34b482: Status 404 returned error can't find the container with id cf0c55d9dab43276787c5b224136d312c14337535bc6854a1ace408a0c34b482
Sep 07 12:16:11.892586 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:11.892487    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq" event={"ID":"d160f8d3-6410-4282-bba6-bd734acbd278","Type":"ContainerStarted","Data":"3ac60e0f3a53b88bb652e45c975a2f8f2b2d0e81191cd3a966ed523b7af98d3f"}
Sep 07 12:16:11.892586 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:11.892537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq" event={"ID":"d160f8d3-6410-4282-bba6-bd734acbd278","Type":"ContainerStarted","Data":"cf0c55d9dab43276787c5b224136d312c14337535bc6854a1ace408a0c34b482"}
Sep 07 12:16:11.907156 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:11.907101    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq" podStartSLOduration=0.907083336 podStartE2EDuration="907.083336ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:16:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:16:11.905837357 +0000 UTC m=+1726.401726781" watchObservedRunningTime="2026-09-07 12:16:11.907083336 +0000 UTC m=+1726.402972759"
Sep 07 12:16:12.901015 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:12.900978    2580 generic.go:356] "Generic (PLEG): container finished" podID="d160f8d3-6410-4282-bba6-bd734acbd278" containerID="3ac60e0f3a53b88bb652e45c975a2f8f2b2d0e81191cd3a966ed523b7af98d3f" exitCode=0
Sep 07 12:16:12.901420 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:12.901028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq" event={"ID":"d160f8d3-6410-4282-bba6-bd734acbd278","Type":"ContainerDied","Data":"3ac60e0f3a53b88bb652e45c975a2f8f2b2d0e81191cd3a966ed523b7af98d3f"}
Sep 07 12:16:14.040857 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:14.040831    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq"
Sep 07 12:16:14.107994 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:14.107952    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d160f8d3-6410-4282-bba6-bd734acbd278-maas-api-service-ca\") pod \"d160f8d3-6410-4282-bba6-bd734acbd278\" (UID: \"d160f8d3-6410-4282-bba6-bd734acbd278\") "
Sep 07 12:16:14.108328 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:14.108297    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d160f8d3-6410-4282-bba6-bd734acbd278-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d160f8d3-6410-4282-bba6-bd734acbd278" (UID: "d160f8d3-6410-4282-bba6-bd734acbd278"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:16:14.108487 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:14.108391    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d160f8d3-6410-4282-bba6-bd734acbd278-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:16:14.914144 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:14.914113    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq"
Sep 07 12:16:14.914342 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:14.914151    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq" event={"ID":"d160f8d3-6410-4282-bba6-bd734acbd278","Type":"ContainerDied","Data":"cf0c55d9dab43276787c5b224136d312c14337535bc6854a1ace408a0c34b482"}
Sep 07 12:16:14.914342 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:14.914223    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf0c55d9dab43276787c5b224136d312c14337535bc6854a1ace408a0c34b482"
Sep 07 12:16:15.902140 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:15.902111    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:16:15.922642 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:15.922550    2580 generic.go:356] "Generic (PLEG): container finished" podID="3077305f-0902-4007-a3e6-beabfbb2a12d" containerID="e58febf34d8e6a5136fcf0d494f29185fd11f9c5355e3765c9eeb289ecacfe4b" exitCode=0
Sep 07 12:16:15.922642 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:15.922613    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"
Sep 07 12:16:15.922642 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:15.922625    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf" event={"ID":"3077305f-0902-4007-a3e6-beabfbb2a12d","Type":"ContainerDied","Data":"e58febf34d8e6a5136fcf0d494f29185fd11f9c5355e3765c9eeb289ecacfe4b"}
Sep 07 12:16:15.922922 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:15.922676    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf" event={"ID":"3077305f-0902-4007-a3e6-beabfbb2a12d","Type":"ContainerDied","Data":"e7b00817fedc3d90440da57d4c465bb5b910de8f4bbd9775af4c563292ed588e"}
Sep 07 12:16:15.922922 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:15.922710    2580 scope.go:117] "RemoveContainer" containerID="e58febf34d8e6a5136fcf0d494f29185fd11f9c5355e3765c9eeb289ecacfe4b"
Sep 07 12:16:15.932214 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:15.932171    2580 scope.go:117] "RemoveContainer" containerID="79050e3f9288b209747c68d170ef93552d85d58f3374f577d5c22f546dfb5bf6"
Sep 07 12:16:15.943539 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:15.943501    2580 scope.go:117] "RemoveContainer" containerID="e58febf34d8e6a5136fcf0d494f29185fd11f9c5355e3765c9eeb289ecacfe4b"
Sep 07 12:16:15.944277 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:16:15.944248    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e58febf34d8e6a5136fcf0d494f29185fd11f9c5355e3765c9eeb289ecacfe4b\": container with ID starting with e58febf34d8e6a5136fcf0d494f29185fd11f9c5355e3765c9eeb289ecacfe4b not found: ID does not exist" containerID="e58febf34d8e6a5136fcf0d494f29185fd11f9c5355e3765c9eeb289ecacfe4b"
Sep 07 12:16:15.944379 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:15.944289    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e58febf34d8e6a5136fcf0d494f29185fd11f9c5355e3765c9eeb289ecacfe4b"} err="failed to get container status \"e58febf34d8e6a5136fcf0d494f29185fd11f9c5355e3765c9eeb289ecacfe4b\": rpc error: code = NotFound desc = could not find container \"e58febf34d8e6a5136fcf0d494f29185fd11f9c5355e3765c9eeb289ecacfe4b\": container with ID starting with e58febf34d8e6a5136fcf0d494f29185fd11f9c5355e3765c9eeb289ecacfe4b not found: ID does not exist"
Sep 07 12:16:15.944379 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:15.944314    2580 scope.go:117] "RemoveContainer" containerID="79050e3f9288b209747c68d170ef93552d85d58f3374f577d5c22f546dfb5bf6"
Sep 07 12:16:15.944723 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:16:15.944697    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79050e3f9288b209747c68d170ef93552d85d58f3374f577d5c22f546dfb5bf6\": container with ID starting with 79050e3f9288b209747c68d170ef93552d85d58f3374f577d5c22f546dfb5bf6 not found: ID does not exist" containerID="79050e3f9288b209747c68d170ef93552d85d58f3374f577d5c22f546dfb5bf6"
Sep 07 12:16:15.944772 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:15.944733    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79050e3f9288b209747c68d170ef93552d85d58f3374f577d5c22f546dfb5bf6"} err="failed to get container status \"79050e3f9288b209747c68d170ef93552d85d58f3374f577d5c22f546dfb5bf6\": rpc error: code = NotFound desc = could not find container \"79050e3f9288b209747c68d170ef93552d85d58f3374f577d5c22f546dfb5bf6\": container with ID starting with 79050e3f9288b209747c68d170ef93552d85d58f3374f577d5c22f546dfb5bf6 not found: ID does not exist"
Sep 07 12:16:16.028601 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.028569    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-podinfo\") pod \"3077305f-0902-4007-a3e6-beabfbb2a12d\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") "
Sep 07 12:16:16.028804 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.028640    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pj9kf\" (UniqueName: \"kubernetes.io/projected/3077305f-0902-4007-a3e6-beabfbb2a12d-kube-api-access-pj9kf\") pod \"3077305f-0902-4007-a3e6-beabfbb2a12d\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") "
Sep 07 12:16:16.028804 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.028670    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-workload-certs\") pod \"3077305f-0902-4007-a3e6-beabfbb2a12d\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") "
Sep 07 12:16:16.028804 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.028712    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-envoy\") pod \"3077305f-0902-4007-a3e6-beabfbb2a12d\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") "
Sep 07 12:16:16.028804 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.028737    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-workload-socket\") pod \"3077305f-0902-4007-a3e6-beabfbb2a12d\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") "
Sep 07 12:16:16.028804 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.028765    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3077305f-0902-4007-a3e6-beabfbb2a12d-istiod-ca-cert\") pod \"3077305f-0902-4007-a3e6-beabfbb2a12d\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") "
Sep 07 12:16:16.029063 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.028807    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-token\") pod \"3077305f-0902-4007-a3e6-beabfbb2a12d\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") "
Sep 07 12:16:16.029063 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.028834    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-credential-socket\") pod \"3077305f-0902-4007-a3e6-beabfbb2a12d\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") "
Sep 07 12:16:16.029063 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.028863    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-data\") pod \"3077305f-0902-4007-a3e6-beabfbb2a12d\" (UID: \"3077305f-0902-4007-a3e6-beabfbb2a12d\") "
Sep 07 12:16:16.029063 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.029005    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3077305f-0902-4007-a3e6-beabfbb2a12d" (UID: "3077305f-0902-4007-a3e6-beabfbb2a12d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:16.029486 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.029081    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3077305f-0902-4007-a3e6-beabfbb2a12d" (UID: "3077305f-0902-4007-a3e6-beabfbb2a12d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:16.029486 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.029186    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3077305f-0902-4007-a3e6-beabfbb2a12d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3077305f-0902-4007-a3e6-beabfbb2a12d" (UID: "3077305f-0902-4007-a3e6-beabfbb2a12d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:16:16.029486 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.029277    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-data" (OuterVolumeSpecName: "istio-data") pod "3077305f-0902-4007-a3e6-beabfbb2a12d" (UID: "3077305f-0902-4007-a3e6-beabfbb2a12d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:16.029486 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.029308    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-workload-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:16:16.029486 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.029331    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-workload-certs\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:16:16.029486 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.029455    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3077305f-0902-4007-a3e6-beabfbb2a12d" (UID: "3077305f-0902-4007-a3e6-beabfbb2a12d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:16.030905 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.030874    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3077305f-0902-4007-a3e6-beabfbb2a12d-kube-api-access-pj9kf" (OuterVolumeSpecName: "kube-api-access-pj9kf") pod "3077305f-0902-4007-a3e6-beabfbb2a12d" (UID: "3077305f-0902-4007-a3e6-beabfbb2a12d"). InnerVolumeSpecName "kube-api-access-pj9kf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:16:16.030905 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.030878    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3077305f-0902-4007-a3e6-beabfbb2a12d" (UID: "3077305f-0902-4007-a3e6-beabfbb2a12d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:16:16.031299 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.031274    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3077305f-0902-4007-a3e6-beabfbb2a12d" (UID: "3077305f-0902-4007-a3e6-beabfbb2a12d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:16:16.031299 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.031290    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-token" (OuterVolumeSpecName: "istio-token") pod "3077305f-0902-4007-a3e6-beabfbb2a12d" (UID: "3077305f-0902-4007-a3e6-beabfbb2a12d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:16:16.130141 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.130094    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj9kf\" (UniqueName: \"kubernetes.io/projected/3077305f-0902-4007-a3e6-beabfbb2a12d-kube-api-access-pj9kf\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:16:16.130141 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.130135    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-envoy\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:16:16.130141 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.130148    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3077305f-0902-4007-a3e6-beabfbb2a12d-istiod-ca-cert\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:16:16.130414 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.130161    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-token\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:16:16.130414 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.130174    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-credential-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:16:16.130414 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.130187    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-data\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:16:16.130414 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.130225    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3077305f-0902-4007-a3e6-beabfbb2a12d-istio-podinfo\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:16:16.245779 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.245746    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"]
Sep 07 12:16:16.249726 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:16.249688    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3a0bb2-openshift-default-744d699866-p5dwf"]
Sep 07 12:16:18.009714 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:18.009670    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3077305f-0902-4007-a3e6-beabfbb2a12d" path="/var/lib/kubelet/pods/3077305f-0902-4007-a3e6-beabfbb2a12d/volumes"
Sep 07 12:16:21.518102 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.518061    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bff0c9-29813055-tvgdv"]
Sep 07 12:16:21.520922 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.520895    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-bff0c9-29813055-tvgdv"]
Sep 07 12:16:21.542752 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.542718    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"]
Sep 07 12:16:21.542940 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.542912    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27" podUID="f1d5239c-d80b-4951-952a-c09711a123f3" containerName="payload-processing" containerID="cri-o://2870fc85e55e19af8d1fce5aad3470c31eb9d1aa109fd8d990d2b9a8afd00e8e" gracePeriod=30
Sep 07 12:16:21.551692 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.551265    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"]
Sep 07 12:16:21.551692 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.551515    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686" podUID="f219ca8d-1c59-48bc-a256-961c65ab899f" containerName="payload-pre-processing" containerID="cri-o://a3b881b0fd4c62d45ec1f0fa90369bc884c14bb7daf2060e52a1affac2a26393" gracePeriod=30
Sep 07 12:16:21.706498 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.706463    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq"]
Sep 07 12:16:21.708187 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.708159    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-bff0c9-ljshq"]
Sep 07 12:16:21.822228 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.822189    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"
Sep 07 12:16:21.841522 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.841497    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"
Sep 07 12:16:21.883494 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.883459    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zltw\" (UniqueName: \"kubernetes.io/projected/f1d5239c-d80b-4951-952a-c09711a123f3-kube-api-access-9zltw\") pod \"f1d5239c-d80b-4951-952a-c09711a123f3\" (UID: \"f1d5239c-d80b-4951-952a-c09711a123f3\") "
Sep 07 12:16:21.883705 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.883527    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1d5239c-d80b-4951-952a-c09711a123f3-plugins-config-volume\") pod \"f1d5239c-d80b-4951-952a-c09711a123f3\" (UID: \"f1d5239c-d80b-4951-952a-c09711a123f3\") "
Sep 07 12:16:21.883934 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.883907    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1d5239c-d80b-4951-952a-c09711a123f3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f1d5239c-d80b-4951-952a-c09711a123f3" (UID: "f1d5239c-d80b-4951-952a-c09711a123f3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:16:21.885646 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.885626    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1d5239c-d80b-4951-952a-c09711a123f3-kube-api-access-9zltw" (OuterVolumeSpecName: "kube-api-access-9zltw") pod "f1d5239c-d80b-4951-952a-c09711a123f3" (UID: "f1d5239c-d80b-4951-952a-c09711a123f3"). InnerVolumeSpecName "kube-api-access-9zltw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:16:21.956131 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.956098    2580 generic.go:356] "Generic (PLEG): container finished" podID="f1d5239c-d80b-4951-952a-c09711a123f3" containerID="2870fc85e55e19af8d1fce5aad3470c31eb9d1aa109fd8d990d2b9a8afd00e8e" exitCode=0
Sep 07 12:16:21.956349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.956142    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"
Sep 07 12:16:21.956349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.956171    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27" event={"ID":"f1d5239c-d80b-4951-952a-c09711a123f3","Type":"ContainerDied","Data":"2870fc85e55e19af8d1fce5aad3470c31eb9d1aa109fd8d990d2b9a8afd00e8e"}
Sep 07 12:16:21.956349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.956255    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27" event={"ID":"f1d5239c-d80b-4951-952a-c09711a123f3","Type":"ContainerDied","Data":"e0b0653ed6a7abf0a465ba6f7af9e114d5772d4d330b250a53201a862e49d0af"}
Sep 07 12:16:21.956349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.956289    2580 scope.go:117] "RemoveContainer" containerID="2870fc85e55e19af8d1fce5aad3470c31eb9d1aa109fd8d990d2b9a8afd00e8e"
Sep 07 12:16:21.957636 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.957606    2580 generic.go:356] "Generic (PLEG): container finished" podID="f219ca8d-1c59-48bc-a256-961c65ab899f" containerID="a3b881b0fd4c62d45ec1f0fa90369bc884c14bb7daf2060e52a1affac2a26393" exitCode=0
Sep 07 12:16:21.957769 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.957641    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"
Sep 07 12:16:21.957769 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.957644    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686" event={"ID":"f219ca8d-1c59-48bc-a256-961c65ab899f","Type":"ContainerDied","Data":"a3b881b0fd4c62d45ec1f0fa90369bc884c14bb7daf2060e52a1affac2a26393"}
Sep 07 12:16:21.957769 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.957739    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686" event={"ID":"f219ca8d-1c59-48bc-a256-961c65ab899f","Type":"ContainerDied","Data":"82694e014f39b523fa5c082685a7e81803bfaa11035ed3b36f2ecebf2a37e6cb"}
Sep 07 12:16:21.965147 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.965132    2580 scope.go:117] "RemoveContainer" containerID="2870fc85e55e19af8d1fce5aad3470c31eb9d1aa109fd8d990d2b9a8afd00e8e"
Sep 07 12:16:21.965426 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:16:21.965410    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2870fc85e55e19af8d1fce5aad3470c31eb9d1aa109fd8d990d2b9a8afd00e8e\": container with ID starting with 2870fc85e55e19af8d1fce5aad3470c31eb9d1aa109fd8d990d2b9a8afd00e8e not found: ID does not exist" containerID="2870fc85e55e19af8d1fce5aad3470c31eb9d1aa109fd8d990d2b9a8afd00e8e"
Sep 07 12:16:21.965493 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.965435    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2870fc85e55e19af8d1fce5aad3470c31eb9d1aa109fd8d990d2b9a8afd00e8e"} err="failed to get container status \"2870fc85e55e19af8d1fce5aad3470c31eb9d1aa109fd8d990d2b9a8afd00e8e\": rpc error: code = NotFound desc = could not find container \"2870fc85e55e19af8d1fce5aad3470c31eb9d1aa109fd8d990d2b9a8afd00e8e\": container with ID starting with 2870fc85e55e19af8d1fce5aad3470c31eb9d1aa109fd8d990d2b9a8afd00e8e not found: ID does not exist"
Sep 07 12:16:21.965493 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.965449    2580 scope.go:117] "RemoveContainer" containerID="a3b881b0fd4c62d45ec1f0fa90369bc884c14bb7daf2060e52a1affac2a26393"
Sep 07 12:16:21.973802 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.973779    2580 scope.go:117] "RemoveContainer" containerID="a3b881b0fd4c62d45ec1f0fa90369bc884c14bb7daf2060e52a1affac2a26393"
Sep 07 12:16:21.974087 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:16:21.974068    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3b881b0fd4c62d45ec1f0fa90369bc884c14bb7daf2060e52a1affac2a26393\": container with ID starting with a3b881b0fd4c62d45ec1f0fa90369bc884c14bb7daf2060e52a1affac2a26393 not found: ID does not exist" containerID="a3b881b0fd4c62d45ec1f0fa90369bc884c14bb7daf2060e52a1affac2a26393"
Sep 07 12:16:21.974152 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.974099    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3b881b0fd4c62d45ec1f0fa90369bc884c14bb7daf2060e52a1affac2a26393"} err="failed to get container status \"a3b881b0fd4c62d45ec1f0fa90369bc884c14bb7daf2060e52a1affac2a26393\": rpc error: code = NotFound desc = could not find container \"a3b881b0fd4c62d45ec1f0fa90369bc884c14bb7daf2060e52a1affac2a26393\": container with ID starting with a3b881b0fd4c62d45ec1f0fa90369bc884c14bb7daf2060e52a1affac2a26393 not found: ID does not exist"
Sep 07 12:16:21.977609 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.977583    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"]
Sep 07 12:16:21.981373 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.981349    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-bff0c9-776579b459-6fr27"]
Sep 07 12:16:21.984458 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.984440    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs6jv\" (UniqueName: \"kubernetes.io/projected/f219ca8d-1c59-48bc-a256-961c65ab899f-kube-api-access-bs6jv\") pod \"f219ca8d-1c59-48bc-a256-961c65ab899f\" (UID: \"f219ca8d-1c59-48bc-a256-961c65ab899f\") "
Sep 07 12:16:21.984552 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.984539    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f219ca8d-1c59-48bc-a256-961c65ab899f-plugins-config-volume\") pod \"f219ca8d-1c59-48bc-a256-961c65ab899f\" (UID: \"f219ca8d-1c59-48bc-a256-961c65ab899f\") "
Sep 07 12:16:21.984750 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.984738    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zltw\" (UniqueName: \"kubernetes.io/projected/f1d5239c-d80b-4951-952a-c09711a123f3-kube-api-access-9zltw\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:16:21.984798 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.984751    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1d5239c-d80b-4951-952a-c09711a123f3-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:16:21.984846 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.984832    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f219ca8d-1c59-48bc-a256-961c65ab899f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f219ca8d-1c59-48bc-a256-961c65ab899f" (UID: "f219ca8d-1c59-48bc-a256-961c65ab899f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:16:21.986599 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:21.986577    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f219ca8d-1c59-48bc-a256-961c65ab899f-kube-api-access-bs6jv" (OuterVolumeSpecName: "kube-api-access-bs6jv") pod "f219ca8d-1c59-48bc-a256-961c65ab899f" (UID: "f219ca8d-1c59-48bc-a256-961c65ab899f"). InnerVolumeSpecName "kube-api-access-bs6jv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:16:22.011479 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:22.011443    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07049ef8-f9a9-42a5-a5fd-56c5bdbad72c" path="/var/lib/kubelet/pods/07049ef8-f9a9-42a5-a5fd-56c5bdbad72c/volumes"
Sep 07 12:16:22.011811 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:22.011789    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d160f8d3-6410-4282-bba6-bd734acbd278" path="/var/lib/kubelet/pods/d160f8d3-6410-4282-bba6-bd734acbd278/volumes"
Sep 07 12:16:22.012301 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:22.012283    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1d5239c-d80b-4951-952a-c09711a123f3" path="/var/lib/kubelet/pods/f1d5239c-d80b-4951-952a-c09711a123f3/volumes"
Sep 07 12:16:22.086333 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:22.086237    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs6jv\" (UniqueName: \"kubernetes.io/projected/f219ca8d-1c59-48bc-a256-961c65ab899f-kube-api-access-bs6jv\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:16:22.086333 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:22.086267    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f219ca8d-1c59-48bc-a256-961c65ab899f-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:16:22.278126 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:22.278093    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"]
Sep 07 12:16:22.281763 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:22.281735    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-bff0c9-745955b6fd-nk686"]
Sep 07 12:16:24.010681 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:24.010640    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f219ca8d-1c59-48bc-a256-961c65ab899f" path="/var/lib/kubelet/pods/f219ca8d-1c59-48bc-a256-961c65ab899f/volumes"
Sep 07 12:16:48.052900 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.052858    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"]
Sep 07 12:16:48.053349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.053246    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f219ca8d-1c59-48bc-a256-961c65ab899f" containerName="payload-pre-processing"
Sep 07 12:16:48.053349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.053259    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f219ca8d-1c59-48bc-a256-961c65ab899f" containerName="payload-pre-processing"
Sep 07 12:16:48.053349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.053273    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d160f8d3-6410-4282-bba6-bd734acbd278" containerName="revoke-keys"
Sep 07 12:16:48.053349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.053279    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d160f8d3-6410-4282-bba6-bd734acbd278" containerName="revoke-keys"
Sep 07 12:16:48.053349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.053286    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3077305f-0902-4007-a3e6-beabfbb2a12d" containerName="istio-proxy"
Sep 07 12:16:48.053349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.053292    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3077305f-0902-4007-a3e6-beabfbb2a12d" containerName="istio-proxy"
Sep 07 12:16:48.053349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.053308    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3077305f-0902-4007-a3e6-beabfbb2a12d" containerName="istio-proxy"
Sep 07 12:16:48.053349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.053316    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3077305f-0902-4007-a3e6-beabfbb2a12d" containerName="istio-proxy"
Sep 07 12:16:48.053349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.053326    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1d5239c-d80b-4951-952a-c09711a123f3" containerName="payload-processing"
Sep 07 12:16:48.053349 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.053331    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1d5239c-d80b-4951-952a-c09711a123f3" containerName="payload-processing"
Sep 07 12:16:48.053708 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.053386    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f219ca8d-1c59-48bc-a256-961c65ab899f" containerName="payload-pre-processing"
Sep 07 12:16:48.053708 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.053397    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3077305f-0902-4007-a3e6-beabfbb2a12d" containerName="istio-proxy"
Sep 07 12:16:48.053708 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.053404    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1d5239c-d80b-4951-952a-c09711a123f3" containerName="payload-processing"
Sep 07 12:16:48.053708 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.053411    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d160f8d3-6410-4282-bba6-bd734acbd278" containerName="revoke-keys"
Sep 07 12:16:48.056631 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.056613    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"
Sep 07 12:16:48.059295 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.059265    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-aab734-dockercfg-n5szq\""
Sep 07 12:16:48.059772 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.059301    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-aab734\""
Sep 07 12:16:48.066802 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.066769    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"]
Sep 07 12:16:48.072877 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.072844    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"]
Sep 07 12:16:48.073653 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.073628    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3077305f-0902-4007-a3e6-beabfbb2a12d" containerName="istio-proxy"
Sep 07 12:16:48.076879 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.076858    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"
Sep 07 12:16:48.088812 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.088472    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"]
Sep 07 12:16:48.133993 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.133947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1352686-ffd9-45cd-a59b-44b5c3b00693-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26\" (UID: \"c1352686-ffd9-45cd-a59b-44b5c3b00693\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"
Sep 07 12:16:48.134158 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.134010    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83537194-3d52-4362-b198-c0e4f8e9aa12-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p\" (UID: \"83537194-3d52-4362-b198-c0e4f8e9aa12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"
Sep 07 12:16:48.134277 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.134181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqmg7\" (UniqueName: \"kubernetes.io/projected/c1352686-ffd9-45cd-a59b-44b5c3b00693-kube-api-access-sqmg7\") pod \"payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26\" (UID: \"c1352686-ffd9-45cd-a59b-44b5c3b00693\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"
Sep 07 12:16:48.134277 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.134245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrb89\" (UniqueName: \"kubernetes.io/projected/83537194-3d52-4362-b198-c0e4f8e9aa12-kube-api-access-nrb89\") pod \"payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p\" (UID: \"83537194-3d52-4362-b198-c0e4f8e9aa12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"
Sep 07 12:16:48.235143 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.235099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqmg7\" (UniqueName: \"kubernetes.io/projected/c1352686-ffd9-45cd-a59b-44b5c3b00693-kube-api-access-sqmg7\") pod \"payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26\" (UID: \"c1352686-ffd9-45cd-a59b-44b5c3b00693\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"
Sep 07 12:16:48.235366 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.235163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrb89\" (UniqueName: \"kubernetes.io/projected/83537194-3d52-4362-b198-c0e4f8e9aa12-kube-api-access-nrb89\") pod \"payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p\" (UID: \"83537194-3d52-4362-b198-c0e4f8e9aa12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"
Sep 07 12:16:48.235366 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.235240    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1352686-ffd9-45cd-a59b-44b5c3b00693-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26\" (UID: \"c1352686-ffd9-45cd-a59b-44b5c3b00693\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"
Sep 07 12:16:48.235366 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.235281    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83537194-3d52-4362-b198-c0e4f8e9aa12-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p\" (UID: \"83537194-3d52-4362-b198-c0e4f8e9aa12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"
Sep 07 12:16:48.236079 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.236049    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1352686-ffd9-45cd-a59b-44b5c3b00693-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26\" (UID: \"c1352686-ffd9-45cd-a59b-44b5c3b00693\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"
Sep 07 12:16:48.236176 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.236134    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83537194-3d52-4362-b198-c0e4f8e9aa12-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p\" (UID: \"83537194-3d52-4362-b198-c0e4f8e9aa12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"
Sep 07 12:16:48.243562 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.243528    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrb89\" (UniqueName: \"kubernetes.io/projected/83537194-3d52-4362-b198-c0e4f8e9aa12-kube-api-access-nrb89\") pod \"payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p\" (UID: \"83537194-3d52-4362-b198-c0e4f8e9aa12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"
Sep 07 12:16:48.243685 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.243642    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqmg7\" (UniqueName: \"kubernetes.io/projected/c1352686-ffd9-45cd-a59b-44b5c3b00693-kube-api-access-sqmg7\") pod \"payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26\" (UID: \"c1352686-ffd9-45cd-a59b-44b5c3b00693\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"
Sep 07 12:16:48.369216 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.369174    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"
Sep 07 12:16:48.390442 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.390410    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"
Sep 07 12:16:48.643877 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.643815    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"]
Sep 07 12:16:48.675325 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:48.675300    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"]
Sep 07 12:16:48.682368 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:16:48.682331    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1352686_ffd9_45cd_a59b_44b5c3b00693.slice/crio-aa9926eaa34a5d8733238a75fb8981a9a7e674cfd1b11967dc911eaa9ba893c1 WatchSource:0}: Error finding container aa9926eaa34a5d8733238a75fb8981a9a7e674cfd1b11967dc911eaa9ba893c1: Status 404 returned error can't find the container with id aa9926eaa34a5d8733238a75fb8981a9a7e674cfd1b11967dc911eaa9ba893c1
Sep 07 12:16:49.105485 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:49.105384    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p" event={"ID":"83537194-3d52-4362-b198-c0e4f8e9aa12","Type":"ContainerStarted","Data":"86ba4a47d318df8b502a25709050a524bc00ec3c84e65433a5ce6593c043eb50"}
Sep 07 12:16:49.105485 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:49.105453    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"
Sep 07 12:16:49.105485 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:49.105476    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p" event={"ID":"83537194-3d52-4362-b198-c0e4f8e9aa12","Type":"ContainerStarted","Data":"000222cf7c1938b6c0610f3b2f1bd7e8466e8a160f8add27ff6e1f944ed1b054"}
Sep 07 12:16:49.107041 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:49.107018    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26" event={"ID":"c1352686-ffd9-45cd-a59b-44b5c3b00693","Type":"ContainerStarted","Data":"f4c71dc2d328b9cdf8fc7f5d48a76ce90cba982b6850486ccf26063d6bc96bf8"}
Sep 07 12:16:49.107167 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:49.107066    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26" event={"ID":"c1352686-ffd9-45cd-a59b-44b5c3b00693","Type":"ContainerStarted","Data":"aa9926eaa34a5d8733238a75fb8981a9a7e674cfd1b11967dc911eaa9ba893c1"}
Sep 07 12:16:49.107245 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:49.107229    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"
Sep 07 12:16:49.121477 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:49.121428    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p" podStartSLOduration=1.121413487 podStartE2EDuration="1.121413487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:16:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:16:49.120289932 +0000 UTC m=+1763.616179354" watchObservedRunningTime="2026-09-07 12:16:49.121413487 +0000 UTC m=+1763.617302903"
Sep 07 12:16:49.136455 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:16:49.136402    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26" podStartSLOduration=1.136384249 podStartE2EDuration="1.136384249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:16:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:16:49.134599264 +0000 UTC m=+1763.630488713" watchObservedRunningTime="2026-09-07 12:16:49.136384249 +0000 UTC m=+1763.632273672"
Sep 07 12:17:00.116770 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:00.116737    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"
Sep 07 12:17:00.117298 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:00.116811    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"
Sep 07 12:17:17.213453 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:17.213415    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"]
Sep 07 12:17:17.216989 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:17.216970    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"
Sep 07 12:17:17.219602 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:17.219577    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-3e7ca1-dockercfg-98t2z\""
Sep 07 12:17:17.219680 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:17.219644    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-3e7ca1\""
Sep 07 12:17:17.221085 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:17.221060    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqnrp\" (UniqueName: \"kubernetes.io/projected/32a9298e-e5d5-47b5-bec3-04371c125416-kube-api-access-rqnrp\") pod \"payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m\" (UID: \"32a9298e-e5d5-47b5-bec3-04371c125416\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"
Sep 07 12:17:17.221206 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:17.221176    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32a9298e-e5d5-47b5-bec3-04371c125416-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m\" (UID: \"32a9298e-e5d5-47b5-bec3-04371c125416\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"
Sep 07 12:17:17.227768 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:17.227737    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"]
Sep 07 12:17:17.324960 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:17.322286    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqnrp\" (UniqueName: \"kubernetes.io/projected/32a9298e-e5d5-47b5-bec3-04371c125416-kube-api-access-rqnrp\") pod \"payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m\" (UID: \"32a9298e-e5d5-47b5-bec3-04371c125416\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"
Sep 07 12:17:17.324960 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:17.322400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32a9298e-e5d5-47b5-bec3-04371c125416-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m\" (UID: \"32a9298e-e5d5-47b5-bec3-04371c125416\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"
Sep 07 12:17:17.324960 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:17.323353    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32a9298e-e5d5-47b5-bec3-04371c125416-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m\" (UID: \"32a9298e-e5d5-47b5-bec3-04371c125416\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"
Sep 07 12:17:17.334059 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:17.332996    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqnrp\" (UniqueName: \"kubernetes.io/projected/32a9298e-e5d5-47b5-bec3-04371c125416-kube-api-access-rqnrp\") pod \"payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m\" (UID: \"32a9298e-e5d5-47b5-bec3-04371c125416\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"
Sep 07 12:17:17.530285 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:17.530179    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"
Sep 07 12:17:17.808326 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:17.808299    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"]
Sep 07 12:17:17.810863 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:17:17.810836    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a9298e_e5d5_47b5_bec3_04371c125416.slice/crio-9a5be875c637c1936cb4f59b4db47101be2e668e370332ecccce604680a026ca WatchSource:0}: Error finding container 9a5be875c637c1936cb4f59b4db47101be2e668e370332ecccce604680a026ca: Status 404 returned error can't find the container with id 9a5be875c637c1936cb4f59b4db47101be2e668e370332ecccce604680a026ca
Sep 07 12:17:18.260254 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:18.260189    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m" event={"ID":"32a9298e-e5d5-47b5-bec3-04371c125416","Type":"ContainerStarted","Data":"e07d17e722d3eb752d22f991930b34e5febeaf2da219c10f3679645053ad94aa"}
Sep 07 12:17:18.260254 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:18.260254    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m" event={"ID":"32a9298e-e5d5-47b5-bec3-04371c125416","Type":"ContainerStarted","Data":"9a5be875c637c1936cb4f59b4db47101be2e668e370332ecccce604680a026ca"}
Sep 07 12:17:18.260792 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:18.260293    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"
Sep 07 12:17:18.276810 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:18.276754    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m" podStartSLOduration=1.276737198 podStartE2EDuration="1.276737198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:17:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:17:18.275225616 +0000 UTC m=+1792.771115035" watchObservedRunningTime="2026-09-07 12:17:18.276737198 +0000 UTC m=+1792.772626620"
Sep 07 12:17:29.268414 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:29.268383    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"
Sep 07 12:17:45.251777 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:45.251744    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-5kjj5_8eaded52-ec7d-4d87-af2f-cd8e7cfe2049/payload-processing/0.log"
Sep 07 12:17:45.367863 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:17:45.367829    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m_32a9298e-e5d5-47b5-bec3-04371c125416/payload-processing/0.log"
Sep 07 12:18:22.981291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:22.981250    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26_c1352686-ffd9-45cd-a59b-44b5c3b00693/payload-processing/0.log"
Sep 07 12:18:23.103752 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:23.103711    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-5kjj5_8eaded52-ec7d-4d87-af2f-cd8e7cfe2049/payload-processing/0.log"
Sep 07 12:18:23.217779 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:23.217747    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m_32a9298e-e5d5-47b5-bec3-04371c125416/payload-processing/0.log"
Sep 07 12:18:26.319519 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:26.319486    2580 scope.go:117] "RemoveContainer" containerID="7df2555669fb8a11ee053c705ff949a5b50d2580742487030e7bd80ddf3a2ece"
Sep 07 12:18:28.526740 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:28.526709    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"]
Sep 07 12:18:28.529987 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:28.529965    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"
Sep 07 12:18:28.532777 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:28.532758    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-845d36\""
Sep 07 12:18:28.532777 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:28.532764    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-845d36-dockercfg-p4z9w\""
Sep 07 12:18:28.540376 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:28.540348    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"]
Sep 07 12:18:28.605314 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:28.605265    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/297e439c-22f6-428f-8039-f95acfebd362-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4\" (UID: \"297e439c-22f6-428f-8039-f95acfebd362\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"
Sep 07 12:18:28.605763 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:28.605668    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcczz\" (UniqueName: \"kubernetes.io/projected/297e439c-22f6-428f-8039-f95acfebd362-kube-api-access-mcczz\") pod \"payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4\" (UID: \"297e439c-22f6-428f-8039-f95acfebd362\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"
Sep 07 12:18:28.707010 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:28.706968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcczz\" (UniqueName: \"kubernetes.io/projected/297e439c-22f6-428f-8039-f95acfebd362-kube-api-access-mcczz\") pod \"payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4\" (UID: \"297e439c-22f6-428f-8039-f95acfebd362\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"
Sep 07 12:18:28.707263 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:28.707085    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/297e439c-22f6-428f-8039-f95acfebd362-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4\" (UID: \"297e439c-22f6-428f-8039-f95acfebd362\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"
Sep 07 12:18:28.707819 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:28.707795    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/297e439c-22f6-428f-8039-f95acfebd362-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4\" (UID: \"297e439c-22f6-428f-8039-f95acfebd362\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"
Sep 07 12:18:28.714736 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:28.714714    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcczz\" (UniqueName: \"kubernetes.io/projected/297e439c-22f6-428f-8039-f95acfebd362-kube-api-access-mcczz\") pod \"payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4\" (UID: \"297e439c-22f6-428f-8039-f95acfebd362\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"
Sep 07 12:18:28.841940 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:28.841832    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"
Sep 07 12:18:29.115244 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:29.115193    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"]
Sep 07 12:18:29.117756 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:18:29.117727    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod297e439c_22f6_428f_8039_f95acfebd362.slice/crio-4ff4fc845ba2f636a4f0e2fff7540570a0a56ab700b452fe844bbbc589663751 WatchSource:0}: Error finding container 4ff4fc845ba2f636a4f0e2fff7540570a0a56ab700b452fe844bbbc589663751: Status 404 returned error can't find the container with id 4ff4fc845ba2f636a4f0e2fff7540570a0a56ab700b452fe844bbbc589663751
Sep 07 12:18:29.638621 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:29.638582    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4" event={"ID":"297e439c-22f6-428f-8039-f95acfebd362","Type":"ContainerStarted","Data":"4e8b7f1074a80500d137af334877059b978e223dc3a04db22eac117ecf74f626"}
Sep 07 12:18:29.639047 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:29.638638    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4" event={"ID":"297e439c-22f6-428f-8039-f95acfebd362","Type":"ContainerStarted","Data":"4ff4fc845ba2f636a4f0e2fff7540570a0a56ab700b452fe844bbbc589663751"}
Sep 07 12:18:29.639047 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:29.638692    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"
Sep 07 12:18:29.655978 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:29.655918    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4" podStartSLOduration=1.655897767 podStartE2EDuration="1.655897767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:18:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:18:29.653035364 +0000 UTC m=+1864.148924799" watchObservedRunningTime="2026-09-07 12:18:29.655897767 +0000 UTC m=+1864.151787194"
Sep 07 12:18:40.646380 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:40.646349    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"
Sep 07 12:18:52.372577 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:52.372497    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk"]
Sep 07 12:18:52.376268 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:52.376246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk"
Sep 07 12:18:52.378716 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:52.378687    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:18:52.378716 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:52.378693    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:18:52.383100 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:52.383069    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk"]
Sep 07 12:18:52.441452 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:52.441418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88b91b72-68ac-43b1-9292-fab8bc66d436-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk\" (UID: \"88b91b72-68ac-43b1-9292-fab8bc66d436\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk"
Sep 07 12:18:52.541996 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:52.541943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88b91b72-68ac-43b1-9292-fab8bc66d436-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk\" (UID: \"88b91b72-68ac-43b1-9292-fab8bc66d436\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk"
Sep 07 12:18:52.542612 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:52.542590    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88b91b72-68ac-43b1-9292-fab8bc66d436-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk\" (UID: \"88b91b72-68ac-43b1-9292-fab8bc66d436\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk"
Sep 07 12:18:52.688038 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:52.687958    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk"
Sep 07 12:18:52.956907 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:52.956739    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk"]
Sep 07 12:18:52.959745 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:18:52.959716    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88b91b72_68ac_43b1_9292_fab8bc66d436.slice/crio-e2adba3d405525c3b26649976fc76b488b57bf725675594f2bb1053c6b287081 WatchSource:0}: Error finding container e2adba3d405525c3b26649976fc76b488b57bf725675594f2bb1053c6b287081: Status 404 returned error can't find the container with id e2adba3d405525c3b26649976fc76b488b57bf725675594f2bb1053c6b287081
Sep 07 12:18:53.765935 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:53.765902    2580 generic.go:356] "Generic (PLEG): container finished" podID="88b91b72-68ac-43b1-9292-fab8bc66d436" containerID="68f944ce750bf9320ce661e0f2d13dc91b6e0481e605ba9cd5fbc51f2887d803" exitCode=0
Sep 07 12:18:53.766362 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:53.765988    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk" event={"ID":"88b91b72-68ac-43b1-9292-fab8bc66d436","Type":"ContainerDied","Data":"68f944ce750bf9320ce661e0f2d13dc91b6e0481e605ba9cd5fbc51f2887d803"}
Sep 07 12:18:53.766362 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:53.766030    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk" event={"ID":"88b91b72-68ac-43b1-9292-fab8bc66d436","Type":"ContainerStarted","Data":"e2adba3d405525c3b26649976fc76b488b57bf725675594f2bb1053c6b287081"}
Sep 07 12:18:54.907410 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:54.907381    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk"
Sep 07 12:18:54.965896 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:54.965859    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88b91b72-68ac-43b1-9292-fab8bc66d436-maas-api-service-ca\") pod \"88b91b72-68ac-43b1-9292-fab8bc66d436\" (UID: \"88b91b72-68ac-43b1-9292-fab8bc66d436\") "
Sep 07 12:18:54.966232 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:54.966185    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88b91b72-68ac-43b1-9292-fab8bc66d436-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "88b91b72-68ac-43b1-9292-fab8bc66d436" (UID: "88b91b72-68ac-43b1-9292-fab8bc66d436"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:18:55.067599 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:55.067506    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88b91b72-68ac-43b1-9292-fab8bc66d436-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:18:55.778046 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:55.778007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk" event={"ID":"88b91b72-68ac-43b1-9292-fab8bc66d436","Type":"ContainerDied","Data":"e2adba3d405525c3b26649976fc76b488b57bf725675594f2bb1053c6b287081"}
Sep 07 12:18:55.778234 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:55.778082    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2adba3d405525c3b26649976fc76b488b57bf725675594f2bb1053c6b287081"
Sep 07 12:18:55.778234 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:55.778021    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk"
Sep 07 12:18:58.425557 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.425525    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"]
Sep 07 12:18:58.425976 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.425740    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4" podUID="297e439c-22f6-428f-8039-f95acfebd362" containerName="payload-processing" containerID="cri-o://4e8b7f1074a80500d137af334877059b978e223dc3a04db22eac117ecf74f626" gracePeriod=30
Sep 07 12:18:58.591559 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.591522    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk"]
Sep 07 12:18:58.594577 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.594556    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-845d36-2m5jk"]
Sep 07 12:18:58.684840 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.684774    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"
Sep 07 12:18:58.795133 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.795099    2580 generic.go:356] "Generic (PLEG): container finished" podID="297e439c-22f6-428f-8039-f95acfebd362" containerID="4e8b7f1074a80500d137af334877059b978e223dc3a04db22eac117ecf74f626" exitCode=0
Sep 07 12:18:58.795333 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.795141    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"
Sep 07 12:18:58.795333 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.795181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4" event={"ID":"297e439c-22f6-428f-8039-f95acfebd362","Type":"ContainerDied","Data":"4e8b7f1074a80500d137af334877059b978e223dc3a04db22eac117ecf74f626"}
Sep 07 12:18:58.795333 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.795263    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4" event={"ID":"297e439c-22f6-428f-8039-f95acfebd362","Type":"ContainerDied","Data":"4ff4fc845ba2f636a4f0e2fff7540570a0a56ab700b452fe844bbbc589663751"}
Sep 07 12:18:58.795333 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.795287    2580 scope.go:117] "RemoveContainer" containerID="4e8b7f1074a80500d137af334877059b978e223dc3a04db22eac117ecf74f626"
Sep 07 12:18:58.804020 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.803999    2580 scope.go:117] "RemoveContainer" containerID="4e8b7f1074a80500d137af334877059b978e223dc3a04db22eac117ecf74f626"
Sep 07 12:18:58.804360 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:18:58.804338    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e8b7f1074a80500d137af334877059b978e223dc3a04db22eac117ecf74f626\": container with ID starting with 4e8b7f1074a80500d137af334877059b978e223dc3a04db22eac117ecf74f626 not found: ID does not exist" containerID="4e8b7f1074a80500d137af334877059b978e223dc3a04db22eac117ecf74f626"
Sep 07 12:18:58.804449 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.804373    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e8b7f1074a80500d137af334877059b978e223dc3a04db22eac117ecf74f626"} err="failed to get container status \"4e8b7f1074a80500d137af334877059b978e223dc3a04db22eac117ecf74f626\": rpc error: code = NotFound desc = could not find container \"4e8b7f1074a80500d137af334877059b978e223dc3a04db22eac117ecf74f626\": container with ID starting with 4e8b7f1074a80500d137af334877059b978e223dc3a04db22eac117ecf74f626 not found: ID does not exist"
Sep 07 12:18:58.807743 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.807724    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/297e439c-22f6-428f-8039-f95acfebd362-plugins-config-volume\") pod \"297e439c-22f6-428f-8039-f95acfebd362\" (UID: \"297e439c-22f6-428f-8039-f95acfebd362\") "
Sep 07 12:18:58.807785 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.807767    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcczz\" (UniqueName: \"kubernetes.io/projected/297e439c-22f6-428f-8039-f95acfebd362-kube-api-access-mcczz\") pod \"297e439c-22f6-428f-8039-f95acfebd362\" (UID: \"297e439c-22f6-428f-8039-f95acfebd362\") "
Sep 07 12:18:58.808142 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.808121    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/297e439c-22f6-428f-8039-f95acfebd362-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "297e439c-22f6-428f-8039-f95acfebd362" (UID: "297e439c-22f6-428f-8039-f95acfebd362"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:18:58.809839 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.809822    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/297e439c-22f6-428f-8039-f95acfebd362-kube-api-access-mcczz" (OuterVolumeSpecName: "kube-api-access-mcczz") pod "297e439c-22f6-428f-8039-f95acfebd362" (UID: "297e439c-22f6-428f-8039-f95acfebd362"). InnerVolumeSpecName "kube-api-access-mcczz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:18:58.908824 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.908788    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/297e439c-22f6-428f-8039-f95acfebd362-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:18:58.908824 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:58.908821    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcczz\" (UniqueName: \"kubernetes.io/projected/297e439c-22f6-428f-8039-f95acfebd362-kube-api-access-mcczz\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:18:59.117159 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:59.117108    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"]
Sep 07 12:18:59.118868 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:18:59.118839    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-845d36-797857b5dc-dgfn4"]
Sep 07 12:19:00.010356 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:00.010319    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="297e439c-22f6-428f-8039-f95acfebd362" path="/var/lib/kubelet/pods/297e439c-22f6-428f-8039-f95acfebd362/volumes"
Sep 07 12:19:00.010736 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:00.010668    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88b91b72-68ac-43b1-9292-fab8bc66d436" path="/var/lib/kubelet/pods/88b91b72-68ac-43b1-9292-fab8bc66d436/volumes"
Sep 07 12:19:08.637256 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:08.637214    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj"]
Sep 07 12:19:08.637751 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:08.637611    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="297e439c-22f6-428f-8039-f95acfebd362" containerName="payload-processing"
Sep 07 12:19:08.637751 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:08.637624    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="297e439c-22f6-428f-8039-f95acfebd362" containerName="payload-processing"
Sep 07 12:19:08.637751 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:08.637640    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88b91b72-68ac-43b1-9292-fab8bc66d436" containerName="revoke-keys"
Sep 07 12:19:08.637751 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:08.637646    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="88b91b72-68ac-43b1-9292-fab8bc66d436" containerName="revoke-keys"
Sep 07 12:19:08.637751 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:08.637724    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="297e439c-22f6-428f-8039-f95acfebd362" containerName="payload-processing"
Sep 07 12:19:08.637751 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:08.637734    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="88b91b72-68ac-43b1-9292-fab8bc66d436" containerName="revoke-keys"
Sep 07 12:19:08.642351 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:08.642333    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj"
Sep 07 12:19:08.645070 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:08.645044    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:19:08.645268 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:08.645045    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:19:08.648586 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:08.648560    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj"]
Sep 07 12:19:08.705290 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:08.705250    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb0a908c-1a84-4d0b-a978-d47d68a53135-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj\" (UID: \"eb0a908c-1a84-4d0b-a978-d47d68a53135\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj"
Sep 07 12:19:08.806683 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:08.806634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb0a908c-1a84-4d0b-a978-d47d68a53135-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj\" (UID: \"eb0a908c-1a84-4d0b-a978-d47d68a53135\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj"
Sep 07 12:19:08.807368 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:08.807342    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb0a908c-1a84-4d0b-a978-d47d68a53135-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj\" (UID: \"eb0a908c-1a84-4d0b-a978-d47d68a53135\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj"
Sep 07 12:19:08.953897 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:08.953810    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj"
Sep 07 12:19:09.224185 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:09.224004    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj"]
Sep 07 12:19:09.226459 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:19:09.226428    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb0a908c_1a84_4d0b_a978_d47d68a53135.slice/crio-aa64549e1fdbbf7d056e0007ec2c4f41ec9e6e99c1d211fb4f1f2e9d98fae806 WatchSource:0}: Error finding container aa64549e1fdbbf7d056e0007ec2c4f41ec9e6e99c1d211fb4f1f2e9d98fae806: Status 404 returned error can't find the container with id aa64549e1fdbbf7d056e0007ec2c4f41ec9e6e99c1d211fb4f1f2e9d98fae806
Sep 07 12:19:09.856760 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:09.856655    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj" event={"ID":"eb0a908c-1a84-4d0b-a978-d47d68a53135","Type":"ContainerStarted","Data":"aeeeee80c9b6bad292bce5aac580a8e18684ec5c45b975f0c9722c41796b4c2f"}
Sep 07 12:19:09.856760 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:09.856709    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj" event={"ID":"eb0a908c-1a84-4d0b-a978-d47d68a53135","Type":"ContainerStarted","Data":"aa64549e1fdbbf7d056e0007ec2c4f41ec9e6e99c1d211fb4f1f2e9d98fae806"}
Sep 07 12:19:09.872593 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:09.872537    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj" podStartSLOduration=1.872518366 podStartE2EDuration="1.872518366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:19:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:19:09.869839802 +0000 UTC m=+1904.365729226" watchObservedRunningTime="2026-09-07 12:19:09.872518366 +0000 UTC m=+1904.368407825"
Sep 07 12:19:10.863335 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:10.863302    2580 generic.go:356] "Generic (PLEG): container finished" podID="eb0a908c-1a84-4d0b-a978-d47d68a53135" containerID="aeeeee80c9b6bad292bce5aac580a8e18684ec5c45b975f0c9722c41796b4c2f" exitCode=0
Sep 07 12:19:10.863730 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:10.863358    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj" event={"ID":"eb0a908c-1a84-4d0b-a978-d47d68a53135","Type":"ContainerDied","Data":"aeeeee80c9b6bad292bce5aac580a8e18684ec5c45b975f0c9722c41796b4c2f"}
Sep 07 12:19:12.008544 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:12.008516    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj"
Sep 07 12:19:12.034719 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:12.034684    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb0a908c-1a84-4d0b-a978-d47d68a53135-maas-api-service-ca\") pod \"eb0a908c-1a84-4d0b-a978-d47d68a53135\" (UID: \"eb0a908c-1a84-4d0b-a978-d47d68a53135\") "
Sep 07 12:19:12.035034 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:12.035002    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb0a908c-1a84-4d0b-a978-d47d68a53135-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eb0a908c-1a84-4d0b-a978-d47d68a53135" (UID: "eb0a908c-1a84-4d0b-a978-d47d68a53135"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:19:12.035643 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:12.035619    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb0a908c-1a84-4d0b-a978-d47d68a53135-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:19:12.875101 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:12.875063    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj" event={"ID":"eb0a908c-1a84-4d0b-a978-d47d68a53135","Type":"ContainerDied","Data":"aa64549e1fdbbf7d056e0007ec2c4f41ec9e6e99c1d211fb4f1f2e9d98fae806"}
Sep 07 12:19:12.875101 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:12.875106    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj"
Sep 07 12:19:12.875358 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:12.875114    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa64549e1fdbbf7d056e0007ec2c4f41ec9e6e99c1d211fb4f1f2e9d98fae806"
Sep 07 12:19:24.295672 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.295628    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"]
Sep 07 12:19:24.296179 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.295894    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26" podUID="c1352686-ffd9-45cd-a59b-44b5c3b00693" containerName="payload-processing" containerID="cri-o://f4c71dc2d328b9cdf8fc7f5d48a76ce90cba982b6850486ccf26063d6bc96bf8" gracePeriod=30
Sep 07 12:19:24.303024 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.302993    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"]
Sep 07 12:19:24.303229 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.303189    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p" podUID="83537194-3d52-4362-b198-c0e4f8e9aa12" containerName="payload-pre-processing" containerID="cri-o://86ba4a47d318df8b502a25709050a524bc00ec3c84e65433a5ce6593c043eb50" gracePeriod=30
Sep 07 12:19:24.480896 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.480860    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj"]
Sep 07 12:19:24.483889 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.483860    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-aab734-rdjsj"]
Sep 07 12:19:24.566509 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.566480    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"
Sep 07 12:19:24.590790 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.590761    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"
Sep 07 12:19:24.647941 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.647905    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrb89\" (UniqueName: \"kubernetes.io/projected/83537194-3d52-4362-b198-c0e4f8e9aa12-kube-api-access-nrb89\") pod \"83537194-3d52-4362-b198-c0e4f8e9aa12\" (UID: \"83537194-3d52-4362-b198-c0e4f8e9aa12\") "
Sep 07 12:19:24.648146 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.647963    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83537194-3d52-4362-b198-c0e4f8e9aa12-plugins-config-volume\") pod \"83537194-3d52-4362-b198-c0e4f8e9aa12\" (UID: \"83537194-3d52-4362-b198-c0e4f8e9aa12\") "
Sep 07 12:19:24.648146 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.648048    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1352686-ffd9-45cd-a59b-44b5c3b00693-plugins-config-volume\") pod \"c1352686-ffd9-45cd-a59b-44b5c3b00693\" (UID: \"c1352686-ffd9-45cd-a59b-44b5c3b00693\") "
Sep 07 12:19:24.648146 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.648122    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqmg7\" (UniqueName: \"kubernetes.io/projected/c1352686-ffd9-45cd-a59b-44b5c3b00693-kube-api-access-sqmg7\") pod \"c1352686-ffd9-45cd-a59b-44b5c3b00693\" (UID: \"c1352686-ffd9-45cd-a59b-44b5c3b00693\") "
Sep 07 12:19:24.648781 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.648733    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83537194-3d52-4362-b198-c0e4f8e9aa12-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "83537194-3d52-4362-b198-c0e4f8e9aa12" (UID: "83537194-3d52-4362-b198-c0e4f8e9aa12"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:19:24.648976 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.648949    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1352686-ffd9-45cd-a59b-44b5c3b00693-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c1352686-ffd9-45cd-a59b-44b5c3b00693" (UID: "c1352686-ffd9-45cd-a59b-44b5c3b00693"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:19:24.650952 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.650917    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83537194-3d52-4362-b198-c0e4f8e9aa12-kube-api-access-nrb89" (OuterVolumeSpecName: "kube-api-access-nrb89") pod "83537194-3d52-4362-b198-c0e4f8e9aa12" (UID: "83537194-3d52-4362-b198-c0e4f8e9aa12"). InnerVolumeSpecName "kube-api-access-nrb89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:19:24.651079 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.651003    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1352686-ffd9-45cd-a59b-44b5c3b00693-kube-api-access-sqmg7" (OuterVolumeSpecName: "kube-api-access-sqmg7") pod "c1352686-ffd9-45cd-a59b-44b5c3b00693" (UID: "c1352686-ffd9-45cd-a59b-44b5c3b00693"). InnerVolumeSpecName "kube-api-access-sqmg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:19:24.749834 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.749792    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrb89\" (UniqueName: \"kubernetes.io/projected/83537194-3d52-4362-b198-c0e4f8e9aa12-kube-api-access-nrb89\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:19:24.749834 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.749827    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83537194-3d52-4362-b198-c0e4f8e9aa12-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:19:24.749834 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.749838    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1352686-ffd9-45cd-a59b-44b5c3b00693-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:19:24.750094 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.749871    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqmg7\" (UniqueName: \"kubernetes.io/projected/c1352686-ffd9-45cd-a59b-44b5c3b00693-kube-api-access-sqmg7\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:19:24.941459 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.941421    2580 generic.go:356] "Generic (PLEG): container finished" podID="83537194-3d52-4362-b198-c0e4f8e9aa12" containerID="86ba4a47d318df8b502a25709050a524bc00ec3c84e65433a5ce6593c043eb50" exitCode=0
Sep 07 12:19:24.941638 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.941468    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"
Sep 07 12:19:24.941638 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.941502    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p" event={"ID":"83537194-3d52-4362-b198-c0e4f8e9aa12","Type":"ContainerDied","Data":"86ba4a47d318df8b502a25709050a524bc00ec3c84e65433a5ce6593c043eb50"}
Sep 07 12:19:24.941638 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.941555    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p" event={"ID":"83537194-3d52-4362-b198-c0e4f8e9aa12","Type":"ContainerDied","Data":"000222cf7c1938b6c0610f3b2f1bd7e8466e8a160f8add27ff6e1f944ed1b054"}
Sep 07 12:19:24.941638 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.941578    2580 scope.go:117] "RemoveContainer" containerID="86ba4a47d318df8b502a25709050a524bc00ec3c84e65433a5ce6593c043eb50"
Sep 07 12:19:24.942979 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.942958    2580 generic.go:356] "Generic (PLEG): container finished" podID="c1352686-ffd9-45cd-a59b-44b5c3b00693" containerID="f4c71dc2d328b9cdf8fc7f5d48a76ce90cba982b6850486ccf26063d6bc96bf8" exitCode=0
Sep 07 12:19:24.943103 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.943005    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"
Sep 07 12:19:24.943103 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.942997    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26" event={"ID":"c1352686-ffd9-45cd-a59b-44b5c3b00693","Type":"ContainerDied","Data":"f4c71dc2d328b9cdf8fc7f5d48a76ce90cba982b6850486ccf26063d6bc96bf8"}
Sep 07 12:19:24.943103 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.943065    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26" event={"ID":"c1352686-ffd9-45cd-a59b-44b5c3b00693","Type":"ContainerDied","Data":"aa9926eaa34a5d8733238a75fb8981a9a7e674cfd1b11967dc911eaa9ba893c1"}
Sep 07 12:19:24.950797 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.950774    2580 scope.go:117] "RemoveContainer" containerID="86ba4a47d318df8b502a25709050a524bc00ec3c84e65433a5ce6593c043eb50"
Sep 07 12:19:24.951182 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:19:24.951161    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86ba4a47d318df8b502a25709050a524bc00ec3c84e65433a5ce6593c043eb50\": container with ID starting with 86ba4a47d318df8b502a25709050a524bc00ec3c84e65433a5ce6593c043eb50 not found: ID does not exist" containerID="86ba4a47d318df8b502a25709050a524bc00ec3c84e65433a5ce6593c043eb50"
Sep 07 12:19:24.951289 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.951190    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86ba4a47d318df8b502a25709050a524bc00ec3c84e65433a5ce6593c043eb50"} err="failed to get container status \"86ba4a47d318df8b502a25709050a524bc00ec3c84e65433a5ce6593c043eb50\": rpc error: code = NotFound desc = could not find container \"86ba4a47d318df8b502a25709050a524bc00ec3c84e65433a5ce6593c043eb50\": container with ID starting with 86ba4a47d318df8b502a25709050a524bc00ec3c84e65433a5ce6593c043eb50 not found: ID does not exist"
Sep 07 12:19:24.951289 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.951239    2580 scope.go:117] "RemoveContainer" containerID="f4c71dc2d328b9cdf8fc7f5d48a76ce90cba982b6850486ccf26063d6bc96bf8"
Sep 07 12:19:24.959908 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.959891    2580 scope.go:117] "RemoveContainer" containerID="f4c71dc2d328b9cdf8fc7f5d48a76ce90cba982b6850486ccf26063d6bc96bf8"
Sep 07 12:19:24.960160 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:19:24.960145    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4c71dc2d328b9cdf8fc7f5d48a76ce90cba982b6850486ccf26063d6bc96bf8\": container with ID starting with f4c71dc2d328b9cdf8fc7f5d48a76ce90cba982b6850486ccf26063d6bc96bf8 not found: ID does not exist" containerID="f4c71dc2d328b9cdf8fc7f5d48a76ce90cba982b6850486ccf26063d6bc96bf8"
Sep 07 12:19:24.960239 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.960164    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4c71dc2d328b9cdf8fc7f5d48a76ce90cba982b6850486ccf26063d6bc96bf8"} err="failed to get container status \"f4c71dc2d328b9cdf8fc7f5d48a76ce90cba982b6850486ccf26063d6bc96bf8\": rpc error: code = NotFound desc = could not find container \"f4c71dc2d328b9cdf8fc7f5d48a76ce90cba982b6850486ccf26063d6bc96bf8\": container with ID starting with f4c71dc2d328b9cdf8fc7f5d48a76ce90cba982b6850486ccf26063d6bc96bf8 not found: ID does not exist"
Sep 07 12:19:24.965281 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.965255    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"]
Sep 07 12:19:24.969533 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.969511    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-aab734-59b84b88df-chz26"]
Sep 07 12:19:24.979378 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.979349    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"]
Sep 07 12:19:24.983189 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:24.983160    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-aab734-55d87c4ccd-tvh9p"]
Sep 07 12:19:26.010825 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:26.010792    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83537194-3d52-4362-b198-c0e4f8e9aa12" path="/var/lib/kubelet/pods/83537194-3d52-4362-b198-c0e4f8e9aa12/volumes"
Sep 07 12:19:26.011222 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:26.011162    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1352686-ffd9-45cd-a59b-44b5c3b00693" path="/var/lib/kubelet/pods/c1352686-ffd9-45cd-a59b-44b5c3b00693/volumes"
Sep 07 12:19:26.011492 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:19:26.011478    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb0a908c-1a84-4d0b-a978-d47d68a53135" path="/var/lib/kubelet/pods/eb0a908c-1a84-4d0b-a978-d47d68a53135/volumes"
Sep 07 12:20:36.247274 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.247185    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr"]
Sep 07 12:20:36.249946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.247546    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb0a908c-1a84-4d0b-a978-d47d68a53135" containerName="revoke-keys"
Sep 07 12:20:36.249946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.247557    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb0a908c-1a84-4d0b-a978-d47d68a53135" containerName="revoke-keys"
Sep 07 12:20:36.249946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.247567    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83537194-3d52-4362-b198-c0e4f8e9aa12" containerName="payload-pre-processing"
Sep 07 12:20:36.249946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.247575    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="83537194-3d52-4362-b198-c0e4f8e9aa12" containerName="payload-pre-processing"
Sep 07 12:20:36.249946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.247596    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1352686-ffd9-45cd-a59b-44b5c3b00693" containerName="payload-processing"
Sep 07 12:20:36.249946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.247602    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1352686-ffd9-45cd-a59b-44b5c3b00693" containerName="payload-processing"
Sep 07 12:20:36.249946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.247653    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb0a908c-1a84-4d0b-a978-d47d68a53135" containerName="revoke-keys"
Sep 07 12:20:36.249946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.247661    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="83537194-3d52-4362-b198-c0e4f8e9aa12" containerName="payload-pre-processing"
Sep 07 12:20:36.249946 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.247670    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1352686-ffd9-45cd-a59b-44b5c3b00693" containerName="payload-processing"
Sep 07 12:20:36.250569 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.250550    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr"
Sep 07 12:20:36.253118 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.253093    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:20:36.253256 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.253141    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:20:36.258719 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.258696    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr"]
Sep 07 12:20:36.345269 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.345234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09d9c782-4765-485e-b4b0-4b4c7c56bb39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr\" (UID: \"09d9c782-4765-485e-b4b0-4b4c7c56bb39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr"
Sep 07 12:20:36.446733 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.446691    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09d9c782-4765-485e-b4b0-4b4c7c56bb39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr\" (UID: \"09d9c782-4765-485e-b4b0-4b4c7c56bb39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr"
Sep 07 12:20:36.447445 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.447384    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09d9c782-4765-485e-b4b0-4b4c7c56bb39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr\" (UID: \"09d9c782-4765-485e-b4b0-4b4c7c56bb39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr"
Sep 07 12:20:36.561417 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.561312    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr"
Sep 07 12:20:36.833156 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:36.833131    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr"]
Sep 07 12:20:36.835350 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:20:36.835321    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09d9c782_4765_485e_b4b0_4b4c7c56bb39.slice/crio-5151ca60eb24e6478f4e676000d9fdf736e0554dfe75f614699e3926d19ff403 WatchSource:0}: Error finding container 5151ca60eb24e6478f4e676000d9fdf736e0554dfe75f614699e3926d19ff403: Status 404 returned error can't find the container with id 5151ca60eb24e6478f4e676000d9fdf736e0554dfe75f614699e3926d19ff403
Sep 07 12:20:37.330072 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:37.330033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr" event={"ID":"09d9c782-4765-485e-b4b0-4b4c7c56bb39","Type":"ContainerStarted","Data":"226cd126fec52b2ce476c270b95c6efaabad9d2adf8594560f225cd668c4a49d"}
Sep 07 12:20:37.330591 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:37.330126    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr" event={"ID":"09d9c782-4765-485e-b4b0-4b4c7c56bb39","Type":"ContainerStarted","Data":"5151ca60eb24e6478f4e676000d9fdf736e0554dfe75f614699e3926d19ff403"}
Sep 07 12:20:37.345038 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:37.344957    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr" podStartSLOduration=1.344940351 podStartE2EDuration="1.344940351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:20:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:20:37.343361102 +0000 UTC m=+1991.839250526" watchObservedRunningTime="2026-09-07 12:20:37.344940351 +0000 UTC m=+1991.840829774"
Sep 07 12:20:38.337789 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:38.337750    2580 generic.go:356] "Generic (PLEG): container finished" podID="09d9c782-4765-485e-b4b0-4b4c7c56bb39" containerID="226cd126fec52b2ce476c270b95c6efaabad9d2adf8594560f225cd668c4a49d" exitCode=0
Sep 07 12:20:38.338191 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:38.337838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr" event={"ID":"09d9c782-4765-485e-b4b0-4b4c7c56bb39","Type":"ContainerDied","Data":"226cd126fec52b2ce476c270b95c6efaabad9d2adf8594560f225cd668c4a49d"}
Sep 07 12:20:39.475384 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:39.475357    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr"
Sep 07 12:20:39.577114 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:39.577074    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09d9c782-4765-485e-b4b0-4b4c7c56bb39-maas-api-service-ca\") pod \"09d9c782-4765-485e-b4b0-4b4c7c56bb39\" (UID: \"09d9c782-4765-485e-b4b0-4b4c7c56bb39\") "
Sep 07 12:20:39.577464 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:39.577441    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09d9c782-4765-485e-b4b0-4b4c7c56bb39-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "09d9c782-4765-485e-b4b0-4b4c7c56bb39" (UID: "09d9c782-4765-485e-b4b0-4b4c7c56bb39"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:20:39.677862 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:39.677806    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09d9c782-4765-485e-b4b0-4b4c7c56bb39-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:20:40.350042 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:40.349956    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr"
Sep 07 12:20:40.350042 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:40.350000    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr" event={"ID":"09d9c782-4765-485e-b4b0-4b4c7c56bb39","Type":"ContainerDied","Data":"5151ca60eb24e6478f4e676000d9fdf736e0554dfe75f614699e3926d19ff403"}
Sep 07 12:20:40.350309 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:40.350047    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5151ca60eb24e6478f4e676000d9fdf736e0554dfe75f614699e3926d19ff403"
Sep 07 12:20:46.915443 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:46.915406    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"]
Sep 07 12:20:46.915851 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:46.915707    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m" podUID="32a9298e-e5d5-47b5-bec3-04371c125416" containerName="payload-processing" containerID="cri-o://e07d17e722d3eb752d22f991930b34e5febeaf2da219c10f3679645053ad94aa" gracePeriod=30
Sep 07 12:20:47.083136 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.083084    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr"]
Sep 07 12:20:47.084678 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.084655    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3e7ca1-7vskr"]
Sep 07 12:20:47.197234 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.193455    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"
Sep 07 12:20:47.363849 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.363798    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32a9298e-e5d5-47b5-bec3-04371c125416-plugins-config-volume\") pod \"32a9298e-e5d5-47b5-bec3-04371c125416\" (UID: \"32a9298e-e5d5-47b5-bec3-04371c125416\") "
Sep 07 12:20:47.364042 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.363905    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqnrp\" (UniqueName: \"kubernetes.io/projected/32a9298e-e5d5-47b5-bec3-04371c125416-kube-api-access-rqnrp\") pod \"32a9298e-e5d5-47b5-bec3-04371c125416\" (UID: \"32a9298e-e5d5-47b5-bec3-04371c125416\") "
Sep 07 12:20:47.364293 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.364271    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32a9298e-e5d5-47b5-bec3-04371c125416-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "32a9298e-e5d5-47b5-bec3-04371c125416" (UID: "32a9298e-e5d5-47b5-bec3-04371c125416"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:20:47.366113 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.366086    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32a9298e-e5d5-47b5-bec3-04371c125416-kube-api-access-rqnrp" (OuterVolumeSpecName: "kube-api-access-rqnrp") pod "32a9298e-e5d5-47b5-bec3-04371c125416" (UID: "32a9298e-e5d5-47b5-bec3-04371c125416"). InnerVolumeSpecName "kube-api-access-rqnrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:20:47.388248 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.388181    2580 generic.go:356] "Generic (PLEG): container finished" podID="32a9298e-e5d5-47b5-bec3-04371c125416" containerID="e07d17e722d3eb752d22f991930b34e5febeaf2da219c10f3679645053ad94aa" exitCode=0
Sep 07 12:20:47.388248 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.388243    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m" event={"ID":"32a9298e-e5d5-47b5-bec3-04371c125416","Type":"ContainerDied","Data":"e07d17e722d3eb752d22f991930b34e5febeaf2da219c10f3679645053ad94aa"}
Sep 07 12:20:47.388493 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.388299    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m" event={"ID":"32a9298e-e5d5-47b5-bec3-04371c125416","Type":"ContainerDied","Data":"9a5be875c637c1936cb4f59b4db47101be2e668e370332ecccce604680a026ca"}
Sep 07 12:20:47.388493 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.388254    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"
Sep 07 12:20:47.388493 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.388324    2580 scope.go:117] "RemoveContainer" containerID="e07d17e722d3eb752d22f991930b34e5febeaf2da219c10f3679645053ad94aa"
Sep 07 12:20:47.400449 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.400433    2580 scope.go:117] "RemoveContainer" containerID="e07d17e722d3eb752d22f991930b34e5febeaf2da219c10f3679645053ad94aa"
Sep 07 12:20:47.400891 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:20:47.400867    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e07d17e722d3eb752d22f991930b34e5febeaf2da219c10f3679645053ad94aa\": container with ID starting with e07d17e722d3eb752d22f991930b34e5febeaf2da219c10f3679645053ad94aa not found: ID does not exist" containerID="e07d17e722d3eb752d22f991930b34e5febeaf2da219c10f3679645053ad94aa"
Sep 07 12:20:47.401004 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.400898    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e07d17e722d3eb752d22f991930b34e5febeaf2da219c10f3679645053ad94aa"} err="failed to get container status \"e07d17e722d3eb752d22f991930b34e5febeaf2da219c10f3679645053ad94aa\": rpc error: code = NotFound desc = could not find container \"e07d17e722d3eb752d22f991930b34e5febeaf2da219c10f3679645053ad94aa\": container with ID starting with e07d17e722d3eb752d22f991930b34e5febeaf2da219c10f3679645053ad94aa not found: ID does not exist"
Sep 07 12:20:47.410216 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.410176    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"]
Sep 07 12:20:47.415119 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.415093    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3e7ca1-7458484d9f-t2l8m"]
Sep 07 12:20:47.465048 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.464962    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32a9298e-e5d5-47b5-bec3-04371c125416-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:20:47.465048 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:47.464991    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqnrp\" (UniqueName: \"kubernetes.io/projected/32a9298e-e5d5-47b5-bec3-04371c125416-kube-api-access-rqnrp\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:20:48.010285 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:48.010251    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09d9c782-4765-485e-b4b0-4b4c7c56bb39" path="/var/lib/kubelet/pods/09d9c782-4765-485e-b4b0-4b4c7c56bb39/volumes"
Sep 07 12:20:48.010667 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:48.010610    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32a9298e-e5d5-47b5-bec3-04371c125416" path="/var/lib/kubelet/pods/32a9298e-e5d5-47b5-bec3-04371c125416/volumes"
Sep 07 12:20:53.786495 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:53.786455    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h"]
Sep 07 12:20:53.786904 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:53.786827    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32a9298e-e5d5-47b5-bec3-04371c125416" containerName="payload-processing"
Sep 07 12:20:53.786904 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:53.786839    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="32a9298e-e5d5-47b5-bec3-04371c125416" containerName="payload-processing"
Sep 07 12:20:53.786904 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:53.786854    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09d9c782-4765-485e-b4b0-4b4c7c56bb39" containerName="revoke-keys"
Sep 07 12:20:53.786904 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:53.786860    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="09d9c782-4765-485e-b4b0-4b4c7c56bb39" containerName="revoke-keys"
Sep 07 12:20:53.787054 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:53.786925    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="09d9c782-4765-485e-b4b0-4b4c7c56bb39" containerName="revoke-keys"
Sep 07 12:20:53.787054 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:53.786936    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="32a9298e-e5d5-47b5-bec3-04371c125416" containerName="payload-processing"
Sep 07 12:20:53.790907 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:53.790889    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h"
Sep 07 12:20:53.794275 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:53.794251    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-54hhj\""
Sep 07 12:20:53.794275 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:53.794269    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:20:53.802247 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:53.802216    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h"]
Sep 07 12:20:53.927525 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:53.927483    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29aaed89-4d18-49fd-9bc8-b529312c13f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h\" (UID: \"29aaed89-4d18-49fd-9bc8-b529312c13f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h"
Sep 07 12:20:54.028961 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:54.028908    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29aaed89-4d18-49fd-9bc8-b529312c13f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h\" (UID: \"29aaed89-4d18-49fd-9bc8-b529312c13f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h"
Sep 07 12:20:54.029603 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:54.029578    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29aaed89-4d18-49fd-9bc8-b529312c13f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h\" (UID: \"29aaed89-4d18-49fd-9bc8-b529312c13f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h"
Sep 07 12:20:54.102106 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:54.102019    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h"
Sep 07 12:20:54.371070 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:54.371035    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h"]
Sep 07 12:20:54.373502 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:20:54.373475    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29aaed89_4d18_49fd_9bc8_b529312c13f1.slice/crio-7e8c5dae6392e790cd582dc3551b16b53e91b763301cdbae919b90b3a31f09fe WatchSource:0}: Error finding container 7e8c5dae6392e790cd582dc3551b16b53e91b763301cdbae919b90b3a31f09fe: Status 404 returned error can't find the container with id 7e8c5dae6392e790cd582dc3551b16b53e91b763301cdbae919b90b3a31f09fe
Sep 07 12:20:54.426470 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:54.426438    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h" event={"ID":"29aaed89-4d18-49fd-9bc8-b529312c13f1","Type":"ContainerStarted","Data":"7e8c5dae6392e790cd582dc3551b16b53e91b763301cdbae919b90b3a31f09fe"}
Sep 07 12:20:55.433170 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:55.433134    2580 generic.go:356] "Generic (PLEG): container finished" podID="29aaed89-4d18-49fd-9bc8-b529312c13f1" containerID="c310d420be99da50707c5406733cdf44c07e753ef2b508b8e952cc9743d91bba" exitCode=0
Sep 07 12:20:55.433552 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:55.433182    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h" event={"ID":"29aaed89-4d18-49fd-9bc8-b529312c13f1","Type":"ContainerDied","Data":"c310d420be99da50707c5406733cdf44c07e753ef2b508b8e952cc9743d91bba"}
Sep 07 12:20:56.568622 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:56.568598    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h"
Sep 07 12:20:56.755078 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:56.754970    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29aaed89-4d18-49fd-9bc8-b529312c13f1-maas-api-service-ca\") pod \"29aaed89-4d18-49fd-9bc8-b529312c13f1\" (UID: \"29aaed89-4d18-49fd-9bc8-b529312c13f1\") "
Sep 07 12:20:56.755416 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:56.755396    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29aaed89-4d18-49fd-9bc8-b529312c13f1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "29aaed89-4d18-49fd-9bc8-b529312c13f1" (UID: "29aaed89-4d18-49fd-9bc8-b529312c13f1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:20:56.856043 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:56.856001    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29aaed89-4d18-49fd-9bc8-b529312c13f1-maas-api-service-ca\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:20:57.444994 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:57.444959    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h"
Sep 07 12:20:57.444994 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:57.444986    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h" event={"ID":"29aaed89-4d18-49fd-9bc8-b529312c13f1","Type":"ContainerDied","Data":"7e8c5dae6392e790cd582dc3551b16b53e91b763301cdbae919b90b3a31f09fe"}
Sep 07 12:20:57.445221 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:20:57.445029    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e8c5dae6392e790cd582dc3551b16b53e91b763301cdbae919b90b3a31f09fe"
Sep 07 12:21:04.312420 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:04.312373    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"]
Sep 07 12:21:04.312893 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:04.312651    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh" podUID="e2b2c6f4-80ca-435f-98fe-4eca94dca98a" containerName="payload-pre-processing" containerID="cri-o://d526ea8152c5a37f345f03ecc1972affa27bf932979904ee76d69bdbee90e19a" gracePeriod=30
Sep 07 12:21:04.480319 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:04.480282    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h"]
Sep 07 12:21:04.485510 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:04.485483    2580 generic.go:356] "Generic (PLEG): container finished" podID="e2b2c6f4-80ca-435f-98fe-4eca94dca98a" containerID="d526ea8152c5a37f345f03ecc1972affa27bf932979904ee76d69bdbee90e19a" exitCode=0
Sep 07 12:21:04.485660 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:04.485576    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh" event={"ID":"e2b2c6f4-80ca-435f-98fe-4eca94dca98a","Type":"ContainerDied","Data":"d526ea8152c5a37f345f03ecc1972affa27bf932979904ee76d69bdbee90e19a"}
Sep 07 12:21:04.485819 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:04.485796    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-491296-z2q7h"]
Sep 07 12:21:04.563714 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:04.563645    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"
Sep 07 12:21:04.623237 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:04.622794    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2b2c6f4-80ca-435f-98fe-4eca94dca98a-plugins-config-volume\") pod \"e2b2c6f4-80ca-435f-98fe-4eca94dca98a\" (UID: \"e2b2c6f4-80ca-435f-98fe-4eca94dca98a\") "
Sep 07 12:21:04.623237 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:04.622922    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shqkz\" (UniqueName: \"kubernetes.io/projected/e2b2c6f4-80ca-435f-98fe-4eca94dca98a-kube-api-access-shqkz\") pod \"e2b2c6f4-80ca-435f-98fe-4eca94dca98a\" (UID: \"e2b2c6f4-80ca-435f-98fe-4eca94dca98a\") "
Sep 07 12:21:04.623572 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:04.623528    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2b2c6f4-80ca-435f-98fe-4eca94dca98a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e2b2c6f4-80ca-435f-98fe-4eca94dca98a" (UID: "e2b2c6f4-80ca-435f-98fe-4eca94dca98a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:21:04.626395 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:04.626359    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2b2c6f4-80ca-435f-98fe-4eca94dca98a-kube-api-access-shqkz" (OuterVolumeSpecName: "kube-api-access-shqkz") pod "e2b2c6f4-80ca-435f-98fe-4eca94dca98a" (UID: "e2b2c6f4-80ca-435f-98fe-4eca94dca98a"). InnerVolumeSpecName "kube-api-access-shqkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:21:04.724774 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:04.724735    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2b2c6f4-80ca-435f-98fe-4eca94dca98a-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:04.724774 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:04.724767    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shqkz\" (UniqueName: \"kubernetes.io/projected/e2b2c6f4-80ca-435f-98fe-4eca94dca98a-kube-api-access-shqkz\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:05.492285 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:05.492252    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"
Sep 07 12:21:05.492285 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:05.492260    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh" event={"ID":"e2b2c6f4-80ca-435f-98fe-4eca94dca98a","Type":"ContainerDied","Data":"595038f2343e529de26199f40e128b6b6a8341adb8a9fe2a54704acc8aee88fd"}
Sep 07 12:21:05.492755 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:05.492326    2580 scope.go:117] "RemoveContainer" containerID="d526ea8152c5a37f345f03ecc1972affa27bf932979904ee76d69bdbee90e19a"
Sep 07 12:21:05.513893 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:05.513867    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"]
Sep 07 12:21:05.517726 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:05.517689    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-491296-6d95866c6b-4nhqh"]
Sep 07 12:21:06.010585 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:06.010551    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29aaed89-4d18-49fd-9bc8-b529312c13f1" path="/var/lib/kubelet/pods/29aaed89-4d18-49fd-9bc8-b529312c13f1/volumes"
Sep 07 12:21:06.010914 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:06.010901    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2b2c6f4-80ca-435f-98fe-4eca94dca98a" path="/var/lib/kubelet/pods/e2b2c6f4-80ca-435f-98fe-4eca94dca98a/volumes"
Sep 07 12:21:11.118899 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:11.118864    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"]
Sep 07 12:21:11.119453 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:11.119142    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4" podUID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" containerName="istio-proxy" containerID="cri-o://d1b09100806082c28492da273520c3e74cf0931e5a32358bd8ee74c2b33772ae" gracePeriod=30
Sep 07 12:21:13.485918 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:13.485876    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 12:21:13.486319 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:13.485949    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4" podUID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 12:21:16.363350 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.363323    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:21:16.435628 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.435530    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-workload-socket\") pod \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") "
Sep 07 12:21:16.435628 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.435572    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-data\") pod \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") "
Sep 07 12:21:16.435628 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.435603    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istiod-ca-cert\") pod \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") "
Sep 07 12:21:16.435915 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.435634    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-credential-socket\") pod \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") "
Sep 07 12:21:16.435915 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.435689    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-podinfo\") pod \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") "
Sep 07 12:21:16.435915 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.435838    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-workload-certs\") pod \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") "
Sep 07 12:21:16.435915 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.435844    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-data" (OuterVolumeSpecName: "istio-data") pod "d52eecbf-1cd0-44ec-822b-b31cc579adbb" (UID: "d52eecbf-1cd0-44ec-822b-b31cc579adbb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:21:16.435915 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.435861    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d52eecbf-1cd0-44ec-822b-b31cc579adbb" (UID: "d52eecbf-1cd0-44ec-822b-b31cc579adbb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:21:16.436239 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.435946    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d52eecbf-1cd0-44ec-822b-b31cc579adbb" (UID: "d52eecbf-1cd0-44ec-822b-b31cc579adbb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:21:16.436239 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.435974    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qzfl\" (UniqueName: \"kubernetes.io/projected/d52eecbf-1cd0-44ec-822b-b31cc579adbb-kube-api-access-5qzfl\") pod \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") "
Sep 07 12:21:16.436239 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.436003    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-token\") pod \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") "
Sep 07 12:21:16.436239 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.436038    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-envoy\") pod \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\" (UID: \"d52eecbf-1cd0-44ec-822b-b31cc579adbb\") "
Sep 07 12:21:16.436239 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.436059    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d52eecbf-1cd0-44ec-822b-b31cc579adbb" (UID: "d52eecbf-1cd0-44ec-822b-b31cc579adbb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:21:16.436239 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.436080    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d52eecbf-1cd0-44ec-822b-b31cc579adbb" (UID: "d52eecbf-1cd0-44ec-822b-b31cc579adbb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:21:16.436463 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.436306    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-workload-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:16.436463 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.436329    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-data\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:16.436463 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.436347    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istiod-ca-cert\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:16.436463 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.436359    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-credential-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:16.436463 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.436371    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-workload-certs\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:16.438009 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.437979    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d52eecbf-1cd0-44ec-822b-b31cc579adbb" (UID: "d52eecbf-1cd0-44ec-822b-b31cc579adbb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:21:16.438135 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.438020    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-token" (OuterVolumeSpecName: "istio-token") pod "d52eecbf-1cd0-44ec-822b-b31cc579adbb" (UID: "d52eecbf-1cd0-44ec-822b-b31cc579adbb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:21:16.438440 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.438419    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d52eecbf-1cd0-44ec-822b-b31cc579adbb" (UID: "d52eecbf-1cd0-44ec-822b-b31cc579adbb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:21:16.438524 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.438440    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d52eecbf-1cd0-44ec-822b-b31cc579adbb-kube-api-access-5qzfl" (OuterVolumeSpecName: "kube-api-access-5qzfl") pod "d52eecbf-1cd0-44ec-822b-b31cc579adbb" (UID: "d52eecbf-1cd0-44ec-822b-b31cc579adbb"). InnerVolumeSpecName "kube-api-access-5qzfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:21:16.537510 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.537468    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-token\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:16.537510 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.537498    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-envoy\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:16.537510 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.537510    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d52eecbf-1cd0-44ec-822b-b31cc579adbb-istio-podinfo\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:16.537758 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.537524    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qzfl\" (UniqueName: \"kubernetes.io/projected/d52eecbf-1cd0-44ec-822b-b31cc579adbb-kube-api-access-5qzfl\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:16.550755 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.550721    2580 generic.go:356] "Generic (PLEG): container finished" podID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" containerID="d1b09100806082c28492da273520c3e74cf0931e5a32358bd8ee74c2b33772ae" exitCode=0
Sep 07 12:21:16.550897 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.550796    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"
Sep 07 12:21:16.550897 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.550799    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4" event={"ID":"d52eecbf-1cd0-44ec-822b-b31cc579adbb","Type":"ContainerDied","Data":"d1b09100806082c28492da273520c3e74cf0931e5a32358bd8ee74c2b33772ae"}
Sep 07 12:21:16.550897 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.550882    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4" event={"ID":"d52eecbf-1cd0-44ec-822b-b31cc579adbb","Type":"ContainerDied","Data":"c619b248f196d8002ba4e1a42934ea571b2a05d864cfa5bac0dc3bff176fec0a"}
Sep 07 12:21:16.551006 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.550905    2580 scope.go:117] "RemoveContainer" containerID="d1b09100806082c28492da273520c3e74cf0931e5a32358bd8ee74c2b33772ae"
Sep 07 12:21:16.560627 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.560605    2580 scope.go:117] "RemoveContainer" containerID="279420f47cf89a444c36578368328f5836714fb2f51ecda771b4891727f32e00"
Sep 07 12:21:16.568843 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.568824    2580 scope.go:117] "RemoveContainer" containerID="d1b09100806082c28492da273520c3e74cf0931e5a32358bd8ee74c2b33772ae"
Sep 07 12:21:16.569149 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:21:16.569126    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1b09100806082c28492da273520c3e74cf0931e5a32358bd8ee74c2b33772ae\": container with ID starting with d1b09100806082c28492da273520c3e74cf0931e5a32358bd8ee74c2b33772ae not found: ID does not exist" containerID="d1b09100806082c28492da273520c3e74cf0931e5a32358bd8ee74c2b33772ae"
Sep 07 12:21:16.569246 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.569158    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1b09100806082c28492da273520c3e74cf0931e5a32358bd8ee74c2b33772ae"} err="failed to get container status \"d1b09100806082c28492da273520c3e74cf0931e5a32358bd8ee74c2b33772ae\": rpc error: code = NotFound desc = could not find container \"d1b09100806082c28492da273520c3e74cf0931e5a32358bd8ee74c2b33772ae\": container with ID starting with d1b09100806082c28492da273520c3e74cf0931e5a32358bd8ee74c2b33772ae not found: ID does not exist"
Sep 07 12:21:16.569246 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.569175    2580 scope.go:117] "RemoveContainer" containerID="279420f47cf89a444c36578368328f5836714fb2f51ecda771b4891727f32e00"
Sep 07 12:21:16.569442 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:21:16.569424    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"279420f47cf89a444c36578368328f5836714fb2f51ecda771b4891727f32e00\": container with ID starting with 279420f47cf89a444c36578368328f5836714fb2f51ecda771b4891727f32e00 not found: ID does not exist" containerID="279420f47cf89a444c36578368328f5836714fb2f51ecda771b4891727f32e00"
Sep 07 12:21:16.569488 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.569449    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"279420f47cf89a444c36578368328f5836714fb2f51ecda771b4891727f32e00"} err="failed to get container status \"279420f47cf89a444c36578368328f5836714fb2f51ecda771b4891727f32e00\": rpc error: code = NotFound desc = could not find container \"279420f47cf89a444c36578368328f5836714fb2f51ecda771b4891727f32e00\": container with ID starting with 279420f47cf89a444c36578368328f5836714fb2f51ecda771b4891727f32e00 not found: ID does not exist"
Sep 07 12:21:16.573802 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.573775    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"]
Sep 07 12:21:16.575564 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:16.575540    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-491296-openshift-default-f8894c954-96mc4"]
Sep 07 12:21:18.011074 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:18.011044    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" path="/var/lib/kubelet/pods/d52eecbf-1cd0-44ec-822b-b31cc579adbb/volumes"
Sep 07 12:21:26.365465 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:26.365431    2580 scope.go:117] "RemoveContainer" containerID="f8fb8dd4ecdcfdaf32c9e70ac77f2ddcb755d0ed9adad75302f689839c18ced4"
Sep 07 12:21:26.373924 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:26.373903    2580 scope.go:117] "RemoveContainer" containerID="0cee2927890ccd097010f3aba4eb7fb0ca34acfababfda0785182b8a6cb5850e"
Sep 07 12:21:26.381411 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:26.381390    2580 scope.go:117] "RemoveContainer" containerID="8ad6ca3e42daadcddd410860f8fb358d4b69e61aa7a75ea7c7347b27a7ad5fc4"
Sep 07 12:21:26.388743 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:26.388723    2580 scope.go:117] "RemoveContainer" containerID="c7f54760132a55a92c6cd62f6c6a1f94f5a6be850fe273ecdede70ba86557f6e"
Sep 07 12:21:26.398440 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:26.398419    2580 scope.go:117] "RemoveContainer" containerID="4d839e1c0d9a6bccde7212a5acae83de467e0eb935641faf38022d11a2be87de"
Sep 07 12:21:32.142850 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.142811    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-613886-29813055-8pk62"]
Sep 07 12:21:32.145446 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.145418    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-613886-29813055-8pk62"]
Sep 07 12:21:32.165171 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.165138    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"]
Sep 07 12:21:32.165470 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.165441    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n" podUID="73bbe142-c824-48e6-8807-2d5ae9b606f3" containerName="payload-processing" containerID="cri-o://8b91d926988ce276d0802edb42fa8450a887d3fb4e930e38aef0149b8f8ca271" gracePeriod=30
Sep 07 12:21:32.172977 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.172947    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"]
Sep 07 12:21:32.173193 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.173160    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9" podUID="def4ecac-2bb7-4b60-bb71-2a662aeb57fe" containerName="payload-pre-processing" containerID="cri-o://a0da1daaf1347314fd02ea470ca13eb460bad568f80cc59b02f3a1dab2883335" gracePeriod=30
Sep 07 12:21:32.451233 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.451191    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"
Sep 07 12:21:32.454427 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.454401    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"
Sep 07 12:21:32.589750 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.589718    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73bbe142-c824-48e6-8807-2d5ae9b606f3-plugins-config-volume\") pod \"73bbe142-c824-48e6-8807-2d5ae9b606f3\" (UID: \"73bbe142-c824-48e6-8807-2d5ae9b606f3\") "
Sep 07 12:21:32.589939 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.589769    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfpdx\" (UniqueName: \"kubernetes.io/projected/def4ecac-2bb7-4b60-bb71-2a662aeb57fe-kube-api-access-cfpdx\") pod \"def4ecac-2bb7-4b60-bb71-2a662aeb57fe\" (UID: \"def4ecac-2bb7-4b60-bb71-2a662aeb57fe\") "
Sep 07 12:21:32.589939 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.589816    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/def4ecac-2bb7-4b60-bb71-2a662aeb57fe-plugins-config-volume\") pod \"def4ecac-2bb7-4b60-bb71-2a662aeb57fe\" (UID: \"def4ecac-2bb7-4b60-bb71-2a662aeb57fe\") "
Sep 07 12:21:32.589939 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.589857    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgpgs\" (UniqueName: \"kubernetes.io/projected/73bbe142-c824-48e6-8807-2d5ae9b606f3-kube-api-access-rgpgs\") pod \"73bbe142-c824-48e6-8807-2d5ae9b606f3\" (UID: \"73bbe142-c824-48e6-8807-2d5ae9b606f3\") "
Sep 07 12:21:32.590159 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.590133    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73bbe142-c824-48e6-8807-2d5ae9b606f3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "73bbe142-c824-48e6-8807-2d5ae9b606f3" (UID: "73bbe142-c824-48e6-8807-2d5ae9b606f3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:21:32.590252 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.590176    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/def4ecac-2bb7-4b60-bb71-2a662aeb57fe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "def4ecac-2bb7-4b60-bb71-2a662aeb57fe" (UID: "def4ecac-2bb7-4b60-bb71-2a662aeb57fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:21:32.591916 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.591891    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/def4ecac-2bb7-4b60-bb71-2a662aeb57fe-kube-api-access-cfpdx" (OuterVolumeSpecName: "kube-api-access-cfpdx") pod "def4ecac-2bb7-4b60-bb71-2a662aeb57fe" (UID: "def4ecac-2bb7-4b60-bb71-2a662aeb57fe"). InnerVolumeSpecName "kube-api-access-cfpdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:21:32.592002 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.591923    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73bbe142-c824-48e6-8807-2d5ae9b606f3-kube-api-access-rgpgs" (OuterVolumeSpecName: "kube-api-access-rgpgs") pod "73bbe142-c824-48e6-8807-2d5ae9b606f3" (UID: "73bbe142-c824-48e6-8807-2d5ae9b606f3"). InnerVolumeSpecName "kube-api-access-rgpgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:21:32.630530 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.630498    2580 generic.go:356] "Generic (PLEG): container finished" podID="def4ecac-2bb7-4b60-bb71-2a662aeb57fe" containerID="a0da1daaf1347314fd02ea470ca13eb460bad568f80cc59b02f3a1dab2883335" exitCode=0
Sep 07 12:21:32.630725 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.630543    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"
Sep 07 12:21:32.630725 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.630584    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9" event={"ID":"def4ecac-2bb7-4b60-bb71-2a662aeb57fe","Type":"ContainerDied","Data":"a0da1daaf1347314fd02ea470ca13eb460bad568f80cc59b02f3a1dab2883335"}
Sep 07 12:21:32.630725 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.630646    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9" event={"ID":"def4ecac-2bb7-4b60-bb71-2a662aeb57fe","Type":"ContainerDied","Data":"e457be51cfb488af77714597c363d19958c64c014c5a5404b67e96ff13c1fdce"}
Sep 07 12:21:32.630725 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.630675    2580 scope.go:117] "RemoveContainer" containerID="a0da1daaf1347314fd02ea470ca13eb460bad568f80cc59b02f3a1dab2883335"
Sep 07 12:21:32.631943 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.631919    2580 generic.go:356] "Generic (PLEG): container finished" podID="73bbe142-c824-48e6-8807-2d5ae9b606f3" containerID="8b91d926988ce276d0802edb42fa8450a887d3fb4e930e38aef0149b8f8ca271" exitCode=0
Sep 07 12:21:32.632030 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.631947    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n" event={"ID":"73bbe142-c824-48e6-8807-2d5ae9b606f3","Type":"ContainerDied","Data":"8b91d926988ce276d0802edb42fa8450a887d3fb4e930e38aef0149b8f8ca271"}
Sep 07 12:21:32.632030 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.631973    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"
Sep 07 12:21:32.632030 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.631996    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n" event={"ID":"73bbe142-c824-48e6-8807-2d5ae9b606f3","Type":"ContainerDied","Data":"d13f9923120a25b2bb39eab4b3c91a838a34a9498206015ca6136073b09d96e0"}
Sep 07 12:21:32.643453 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.641361    2580 scope.go:117] "RemoveContainer" containerID="a0da1daaf1347314fd02ea470ca13eb460bad568f80cc59b02f3a1dab2883335"
Sep 07 12:21:32.643688 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:21:32.643456    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0da1daaf1347314fd02ea470ca13eb460bad568f80cc59b02f3a1dab2883335\": container with ID starting with a0da1daaf1347314fd02ea470ca13eb460bad568f80cc59b02f3a1dab2883335 not found: ID does not exist" containerID="a0da1daaf1347314fd02ea470ca13eb460bad568f80cc59b02f3a1dab2883335"
Sep 07 12:21:32.643688 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.643497    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0da1daaf1347314fd02ea470ca13eb460bad568f80cc59b02f3a1dab2883335"} err="failed to get container status \"a0da1daaf1347314fd02ea470ca13eb460bad568f80cc59b02f3a1dab2883335\": rpc error: code = NotFound desc = could not find container \"a0da1daaf1347314fd02ea470ca13eb460bad568f80cc59b02f3a1dab2883335\": container with ID starting with a0da1daaf1347314fd02ea470ca13eb460bad568f80cc59b02f3a1dab2883335 not found: ID does not exist"
Sep 07 12:21:32.643688 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.643520    2580 scope.go:117] "RemoveContainer" containerID="8b91d926988ce276d0802edb42fa8450a887d3fb4e930e38aef0149b8f8ca271"
Sep 07 12:21:32.652241 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.652214    2580 scope.go:117] "RemoveContainer" containerID="8b91d926988ce276d0802edb42fa8450a887d3fb4e930e38aef0149b8f8ca271"
Sep 07 12:21:32.652523 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:21:32.652504    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b91d926988ce276d0802edb42fa8450a887d3fb4e930e38aef0149b8f8ca271\": container with ID starting with 8b91d926988ce276d0802edb42fa8450a887d3fb4e930e38aef0149b8f8ca271 not found: ID does not exist" containerID="8b91d926988ce276d0802edb42fa8450a887d3fb4e930e38aef0149b8f8ca271"
Sep 07 12:21:32.652570 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.652529    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b91d926988ce276d0802edb42fa8450a887d3fb4e930e38aef0149b8f8ca271"} err="failed to get container status \"8b91d926988ce276d0802edb42fa8450a887d3fb4e930e38aef0149b8f8ca271\": rpc error: code = NotFound desc = could not find container \"8b91d926988ce276d0802edb42fa8450a887d3fb4e930e38aef0149b8f8ca271\": container with ID starting with 8b91d926988ce276d0802edb42fa8450a887d3fb4e930e38aef0149b8f8ca271 not found: ID does not exist"
Sep 07 12:21:32.656263 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.656237    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"]
Sep 07 12:21:32.665338 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.660881    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-613886-7f947fd454-679h9"]
Sep 07 12:21:32.675924 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.675840    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"]
Sep 07 12:21:32.681271 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.681238    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-613886-5d6c4b7c4f-v8k7n"]
Sep 07 12:21:32.691478 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.691445    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgpgs\" (UniqueName: \"kubernetes.io/projected/73bbe142-c824-48e6-8807-2d5ae9b606f3-kube-api-access-rgpgs\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:32.691478 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.691472    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73bbe142-c824-48e6-8807-2d5ae9b606f3-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:32.691478 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.691484    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfpdx\" (UniqueName: \"kubernetes.io/projected/def4ecac-2bb7-4b60-bb71-2a662aeb57fe-kube-api-access-cfpdx\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:32.691670 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:32.691492    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/def4ecac-2bb7-4b60-bb71-2a662aeb57fe-plugins-config-volume\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:34.009958 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:34.009923    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73bbe142-c824-48e6-8807-2d5ae9b606f3" path="/var/lib/kubelet/pods/73bbe142-c824-48e6-8807-2d5ae9b606f3/volumes"
Sep 07 12:21:34.010343 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:34.010231    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb7c7e70-3e73-415b-b96b-05ca396b92fc" path="/var/lib/kubelet/pods/bb7c7e70-3e73-415b-b96b-05ca396b92fc/volumes"
Sep 07 12:21:34.010516 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:34.010503    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="def4ecac-2bb7-4b60-bb71-2a662aeb57fe" path="/var/lib/kubelet/pods/def4ecac-2bb7-4b60-bb71-2a662aeb57fe/volumes"
Sep 07 12:21:38.839952 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:38.839918    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"]
Sep 07 12:21:38.840357 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:38.840191    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d" podUID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerName="istio-proxy" containerID="cri-o://a30560e2000e8a8fabe11a52da4d2617c7296e0bb82b396966fdaf0183e28c7d" gracePeriod=30
Sep 07 12:21:43.270504 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:43.270456    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 12:21:43.270953 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:43.270515    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d" podUID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 12:21:44.096178 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.096146    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:21:44.179543 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.179502    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-credential-socket\") pod \"554cfa21-b6f6-44e2-9d05-c46047bb3539\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") "
Sep 07 12:21:44.179776 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.179577    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-podinfo\") pod \"554cfa21-b6f6-44e2-9d05-c46047bb3539\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") "
Sep 07 12:21:44.179776 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.179613    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6g2s6\" (UniqueName: \"kubernetes.io/projected/554cfa21-b6f6-44e2-9d05-c46047bb3539-kube-api-access-6g2s6\") pod \"554cfa21-b6f6-44e2-9d05-c46047bb3539\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") "
Sep 07 12:21:44.179776 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.179640    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-workload-certs\") pod \"554cfa21-b6f6-44e2-9d05-c46047bb3539\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") "
Sep 07 12:21:44.179776 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.179668    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-workload-socket\") pod \"554cfa21-b6f6-44e2-9d05-c46047bb3539\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") "
Sep 07 12:21:44.179776 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.179700    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/554cfa21-b6f6-44e2-9d05-c46047bb3539-istiod-ca-cert\") pod \"554cfa21-b6f6-44e2-9d05-c46047bb3539\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") "
Sep 07 12:21:44.179776 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.179746    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-data\") pod \"554cfa21-b6f6-44e2-9d05-c46047bb3539\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") "
Sep 07 12:21:44.179776 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.179777    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-token\") pod \"554cfa21-b6f6-44e2-9d05-c46047bb3539\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") "
Sep 07 12:21:44.180218 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.179808    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-envoy\") pod \"554cfa21-b6f6-44e2-9d05-c46047bb3539\" (UID: \"554cfa21-b6f6-44e2-9d05-c46047bb3539\") "
Sep 07 12:21:44.180218 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.179888    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "554cfa21-b6f6-44e2-9d05-c46047bb3539" (UID: "554cfa21-b6f6-44e2-9d05-c46047bb3539"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:21:44.180218 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.179927    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "554cfa21-b6f6-44e2-9d05-c46047bb3539" (UID: "554cfa21-b6f6-44e2-9d05-c46047bb3539"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:21:44.180218 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.180134    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-credential-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:44.180218 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.180156    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-workload-certs\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:44.180218 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.180136    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-data" (OuterVolumeSpecName: "istio-data") pod "554cfa21-b6f6-44e2-9d05-c46047bb3539" (UID: "554cfa21-b6f6-44e2-9d05-c46047bb3539"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:21:44.180555 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.180405    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "554cfa21-b6f6-44e2-9d05-c46047bb3539" (UID: "554cfa21-b6f6-44e2-9d05-c46047bb3539"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:21:44.180555 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.180434    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/554cfa21-b6f6-44e2-9d05-c46047bb3539-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "554cfa21-b6f6-44e2-9d05-c46047bb3539" (UID: "554cfa21-b6f6-44e2-9d05-c46047bb3539"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:21:44.182677 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.182609    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "554cfa21-b6f6-44e2-9d05-c46047bb3539" (UID: "554cfa21-b6f6-44e2-9d05-c46047bb3539"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:21:44.182822 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.182701    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-token" (OuterVolumeSpecName: "istio-token") pod "554cfa21-b6f6-44e2-9d05-c46047bb3539" (UID: "554cfa21-b6f6-44e2-9d05-c46047bb3539"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:21:44.182822 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.182785    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/554cfa21-b6f6-44e2-9d05-c46047bb3539-kube-api-access-6g2s6" (OuterVolumeSpecName: "kube-api-access-6g2s6") pod "554cfa21-b6f6-44e2-9d05-c46047bb3539" (UID: "554cfa21-b6f6-44e2-9d05-c46047bb3539"). InnerVolumeSpecName "kube-api-access-6g2s6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:21:44.183070 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.183050    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "554cfa21-b6f6-44e2-9d05-c46047bb3539" (UID: "554cfa21-b6f6-44e2-9d05-c46047bb3539"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 07 12:21:44.280960 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.280921    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-podinfo\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:44.280960 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.280952    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6g2s6\" (UniqueName: \"kubernetes.io/projected/554cfa21-b6f6-44e2-9d05-c46047bb3539-kube-api-access-6g2s6\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:44.280960 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.280962    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-workload-socket\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:44.280960 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.280970    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/554cfa21-b6f6-44e2-9d05-c46047bb3539-istiod-ca-cert\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:44.281661 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.280979    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-data\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:44.281661 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.280987    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-token\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:44.281661 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.280993    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/554cfa21-b6f6-44e2-9d05-c46047bb3539-istio-envoy\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:21:44.690783 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.690747    2580 generic.go:356] "Generic (PLEG): container finished" podID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerID="a30560e2000e8a8fabe11a52da4d2617c7296e0bb82b396966fdaf0183e28c7d" exitCode=0
Sep 07 12:21:44.690968 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.690795    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"
Sep 07 12:21:44.690968 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.690828    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d" event={"ID":"554cfa21-b6f6-44e2-9d05-c46047bb3539","Type":"ContainerDied","Data":"a30560e2000e8a8fabe11a52da4d2617c7296e0bb82b396966fdaf0183e28c7d"}
Sep 07 12:21:44.690968 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.690884    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d" event={"ID":"554cfa21-b6f6-44e2-9d05-c46047bb3539","Type":"ContainerDied","Data":"266d58d38f30836e189dc517c73fec1e0cd6ef90cad561d5ffdc4e60af72a224"}
Sep 07 12:21:44.690968 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.690909    2580 scope.go:117] "RemoveContainer" containerID="a30560e2000e8a8fabe11a52da4d2617c7296e0bb82b396966fdaf0183e28c7d"
Sep 07 12:21:44.699759 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.699738    2580 scope.go:117] "RemoveContainer" containerID="47b8f95fd9a81ae3a9b7446c889c1ea1f9fa045ebe05570616d3aa1acba97c3d"
Sep 07 12:21:44.707426 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.707410    2580 scope.go:117] "RemoveContainer" containerID="a30560e2000e8a8fabe11a52da4d2617c7296e0bb82b396966fdaf0183e28c7d"
Sep 07 12:21:44.707687 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:21:44.707669    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a30560e2000e8a8fabe11a52da4d2617c7296e0bb82b396966fdaf0183e28c7d\": container with ID starting with a30560e2000e8a8fabe11a52da4d2617c7296e0bb82b396966fdaf0183e28c7d not found: ID does not exist" containerID="a30560e2000e8a8fabe11a52da4d2617c7296e0bb82b396966fdaf0183e28c7d"
Sep 07 12:21:44.707758 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.707700    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a30560e2000e8a8fabe11a52da4d2617c7296e0bb82b396966fdaf0183e28c7d"} err="failed to get container status \"a30560e2000e8a8fabe11a52da4d2617c7296e0bb82b396966fdaf0183e28c7d\": rpc error: code = NotFound desc = could not find container \"a30560e2000e8a8fabe11a52da4d2617c7296e0bb82b396966fdaf0183e28c7d\": container with ID starting with a30560e2000e8a8fabe11a52da4d2617c7296e0bb82b396966fdaf0183e28c7d not found: ID does not exist"
Sep 07 12:21:44.707758 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.707724    2580 scope.go:117] "RemoveContainer" containerID="47b8f95fd9a81ae3a9b7446c889c1ea1f9fa045ebe05570616d3aa1acba97c3d"
Sep 07 12:21:44.707988 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:21:44.707970    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47b8f95fd9a81ae3a9b7446c889c1ea1f9fa045ebe05570616d3aa1acba97c3d\": container with ID starting with 47b8f95fd9a81ae3a9b7446c889c1ea1f9fa045ebe05570616d3aa1acba97c3d not found: ID does not exist" containerID="47b8f95fd9a81ae3a9b7446c889c1ea1f9fa045ebe05570616d3aa1acba97c3d"
Sep 07 12:21:44.708029 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.708001    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47b8f95fd9a81ae3a9b7446c889c1ea1f9fa045ebe05570616d3aa1acba97c3d"} err="failed to get container status \"47b8f95fd9a81ae3a9b7446c889c1ea1f9fa045ebe05570616d3aa1acba97c3d\": rpc error: code = NotFound desc = could not find container \"47b8f95fd9a81ae3a9b7446c889c1ea1f9fa045ebe05570616d3aa1acba97c3d\": container with ID starting with 47b8f95fd9a81ae3a9b7446c889c1ea1f9fa045ebe05570616d3aa1acba97c3d not found: ID does not exist"
Sep 07 12:21:44.712539 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.712519    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"]
Sep 07 12:21:44.716485 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:44.716460    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-613886-openshift-default-6fdfcdb786-zxs2d"]
Sep 07 12:21:46.010116 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:21:46.010081    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="554cfa21-b6f6-44e2-9d05-c46047bb3539" path="/var/lib/kubelet/pods/554cfa21-b6f6-44e2-9d05-c46047bb3539/volumes"
Sep 07 12:22:26.443659 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:22:26.443581    2580 scope.go:117] "RemoveContainer" containerID="d824fc593c0c2feb20022fc558f827a611f9e2a70330a0cc11188d7d70e09f4f"
Sep 07 12:22:26.451866 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:22:26.451844    2580 scope.go:117] "RemoveContainer" containerID="ba3c4a2922ca7c7e9a2a55fc12bf4e81a49aa5daac1bfefde84d9202ae066431"
Sep 07 12:22:26.459897 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:22:26.459873    2580 scope.go:117] "RemoveContainer" containerID="3ac60e0f3a53b88bb652e45c975a2f8f2b2d0e81191cd3a966ed523b7af98d3f"
Sep 07 12:22:26.467510 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:22:26.467492    2580 scope.go:117] "RemoveContainer" containerID="533128732d31143730cfabdf295fee156a0e0645328e32cb5c9e207acebf7c91"
Sep 07 12:22:41.180914 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:22:41.180866    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-128-68.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:22:41.248004 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:22:41.247961    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-128-68.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:22:41.271484 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:22:41.271445    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-128-68.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:22:41.289611 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:22:41.289522    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-128-68.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:22:41.306608 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:22:41.306570    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:41Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-128-68.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:22:41.306608 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:22:41.306606    2580 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Sep 07 12:22:51.359901 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:22:51.359853    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-128-68.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:22:51.374567 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:22:51.374528    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-128-68.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:22:51.390633 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:22:51.390587    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-128-68.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:22:51.405514 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:22:51.405478    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-128-68.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:22:51.418609 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:22:51.418563    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:22:51Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-128-68.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:22:51.418609 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:22:51.418601    2580 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Sep 07 12:23:01.587649 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:01.587608    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-128-68.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:23:01.602812 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:01.602761    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-128-68.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:23:01.615215 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:01.615148    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-128-68.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:23:01.628331 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:01.628289    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-128-68.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:23:01.640433 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:01.640385    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-09-07T12:23:01Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"}]}}\" for node \"ip-10-0-128-68.ec2.internal\": Internal error occurred: failed calling webhook \"node.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/node?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused"
Sep 07 12:23:01.640433 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:01.640420    2580 kubelet_node_status.go:523] "Unable to update node status" err="update node status exceeds retry count"
Sep 07 12:23:03.951877 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.951837    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/cni-sysctl-allowlist-ds-vmtpv"]
Sep 07 12:23:03.952291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952134    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73bbe142-c824-48e6-8807-2d5ae9b606f3" containerName="payload-processing"
Sep 07 12:23:03.952291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952145    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="73bbe142-c824-48e6-8807-2d5ae9b606f3" containerName="payload-processing"
Sep 07 12:23:03.952291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952158    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="def4ecac-2bb7-4b60-bb71-2a662aeb57fe" containerName="payload-pre-processing"
Sep 07 12:23:03.952291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952164    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="def4ecac-2bb7-4b60-bb71-2a662aeb57fe" containerName="payload-pre-processing"
Sep 07 12:23:03.952291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952174    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2b2c6f4-80ca-435f-98fe-4eca94dca98a" containerName="payload-pre-processing"
Sep 07 12:23:03.952291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952180    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2b2c6f4-80ca-435f-98fe-4eca94dca98a" containerName="payload-pre-processing"
Sep 07 12:23:03.952291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952186    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerName="istio-proxy"
Sep 07 12:23:03.952291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952192    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerName="istio-proxy"
Sep 07 12:23:03.952291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952214    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29aaed89-4d18-49fd-9bc8-b529312c13f1" containerName="revoke-keys"
Sep 07 12:23:03.952291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952222    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29aaed89-4d18-49fd-9bc8-b529312c13f1" containerName="revoke-keys"
Sep 07 12:23:03.952291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952245    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" containerName="istio-proxy"
Sep 07 12:23:03.952291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952251    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" containerName="istio-proxy"
Sep 07 12:23:03.952291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952262    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" containerName="istio-proxy"
Sep 07 12:23:03.952291 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952267    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" containerName="istio-proxy"
Sep 07 12:23:03.952766 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952313    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" containerName="istio-proxy"
Sep 07 12:23:03.952766 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952321    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerName="istio-proxy"
Sep 07 12:23:03.952766 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952328    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerName="istio-proxy"
Sep 07 12:23:03.952766 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952334    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2b2c6f4-80ca-435f-98fe-4eca94dca98a" containerName="payload-pre-processing"
Sep 07 12:23:03.952766 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952340    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="73bbe142-c824-48e6-8807-2d5ae9b606f3" containerName="payload-processing"
Sep 07 12:23:03.952766 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952344    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="def4ecac-2bb7-4b60-bb71-2a662aeb57fe" containerName="payload-pre-processing"
Sep 07 12:23:03.952766 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.952351    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29aaed89-4d18-49fd-9bc8-b529312c13f1" containerName="revoke-keys"
Sep 07 12:23:03.955172 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.955155    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:03.957611 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:03.957594    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-sysctl-allowlist\""
Sep 07 12:23:03.967612 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:03.967564    2580 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e6fa1e3b-1be4-45bc-9275-84c2d4503196\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:03Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:03Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:03Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"observedGeneration\\\":1,\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:03Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"observedGeneration\\\":1,\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":1,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/ready\\\",\\\"name\\\":\\\"ready\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-pz2bv\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"10.0.128.68\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"10.0.128.68\\\"}],\\\"observedGeneration\\\":1,\\\"podIP\\\":\\\"10.0.128.68\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"10.0.128.68\\\"}],\\\"startTime\\\":\\\"2026-09-07T12:23:03Z\\\"}}\" for pod \"openshift-multus\"/\"cni-sysctl-allowlist-ds-vmtpv\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/pod?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:04.093625 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:04.093587    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ready\" (UniqueName: \"kubernetes.io/empty-dir/e6fa1e3b-1be4-45bc-9275-84c2d4503196-ready\") pod \"cni-sysctl-allowlist-ds-vmtpv\" (UID: \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\") " pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:04.093813 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:04.093638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6fa1e3b-1be4-45bc-9275-84c2d4503196-cni-sysctl-allowlist\") pod \"cni-sysctl-allowlist-ds-vmtpv\" (UID: \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\") " pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:04.093813 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:04.093769    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6fa1e3b-1be4-45bc-9275-84c2d4503196-tuning-conf-dir\") pod \"cni-sysctl-allowlist-ds-vmtpv\" (UID: \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\") " pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:04.093813 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:04.093807    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz2bv\" (UniqueName: \"kubernetes.io/projected/e6fa1e3b-1be4-45bc-9275-84c2d4503196-kube-api-access-pz2bv\") pod \"cni-sysctl-allowlist-ds-vmtpv\" (UID: \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\") " pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:04.194972 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:04.194936    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6fa1e3b-1be4-45bc-9275-84c2d4503196-tuning-conf-dir\") pod \"cni-sysctl-allowlist-ds-vmtpv\" (UID: \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\") " pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:04.195098 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:04.194978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz2bv\" (UniqueName: \"kubernetes.io/projected/e6fa1e3b-1be4-45bc-9275-84c2d4503196-kube-api-access-pz2bv\") pod \"cni-sysctl-allowlist-ds-vmtpv\" (UID: \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\") " pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:04.195098 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:04.195021    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ready\" (UniqueName: \"kubernetes.io/empty-dir/e6fa1e3b-1be4-45bc-9275-84c2d4503196-ready\") pod \"cni-sysctl-allowlist-ds-vmtpv\" (UID: \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\") " pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:04.195098 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:04.195045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6fa1e3b-1be4-45bc-9275-84c2d4503196-cni-sysctl-allowlist\") pod \"cni-sysctl-allowlist-ds-vmtpv\" (UID: \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\") " pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:04.195098 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:04.195066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6fa1e3b-1be4-45bc-9275-84c2d4503196-tuning-conf-dir\") pod \"cni-sysctl-allowlist-ds-vmtpv\" (UID: \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\") " pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:04.195475 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:04.195456    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ready\" (UniqueName: \"kubernetes.io/empty-dir/e6fa1e3b-1be4-45bc-9275-84c2d4503196-ready\") pod \"cni-sysctl-allowlist-ds-vmtpv\" (UID: \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\") " pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:04.195574 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:04.195559    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6fa1e3b-1be4-45bc-9275-84c2d4503196-cni-sysctl-allowlist\") pod \"cni-sysctl-allowlist-ds-vmtpv\" (UID: \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\") " pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:04.202259 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:04.202210    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz2bv\" (UniqueName: \"kubernetes.io/projected/e6fa1e3b-1be4-45bc-9275-84c2d4503196-kube-api-access-pz2bv\") pod \"cni-sysctl-allowlist-ds-vmtpv\" (UID: \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\") " pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:04.264556 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:04.264518    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:04.277939 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:23:04.277904    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6fa1e3b_1be4_45bc_9275_84c2d4503196.slice/crio-4e15e1c2ab6eca780aa547c56a2c8c8c0e614a99327fcafde8768c877f928176 WatchSource:0}: Error finding container 4e15e1c2ab6eca780aa547c56a2c8c8c0e614a99327fcafde8768c877f928176: Status 404 returned error can't find the container with id 4e15e1c2ab6eca780aa547c56a2c8c8c0e614a99327fcafde8768c877f928176
Sep 07 12:23:05.052267 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:05.052232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv" event={"ID":"e6fa1e3b-1be4-45bc-9275-84c2d4503196","Type":"ContainerStarted","Data":"ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc"}
Sep 07 12:23:05.052670 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:05.052288    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv" event={"ID":"e6fa1e3b-1be4-45bc-9275-84c2d4503196","Type":"ContainerStarted","Data":"4e15e1c2ab6eca780aa547c56a2c8c8c0e614a99327fcafde8768c877f928176"}
Sep 07 12:23:05.052670 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:05.052512    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:05.062456 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:05.062394    2580 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e6fa1e3b-1be4-45bc-9275-84c2d4503196\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:05Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:03Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:03Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"observedGeneration\\\":1,\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:03Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"observedGeneration\\\":1,\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":1,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-09-07T12:23:04Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/ready\\\",\\\"name\\\":\\\"ready\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-pz2bv\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"10.0.128.68\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"10.0.128.68\\\"}],\\\"observedGeneration\\\":1,\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"10.0.128.68\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"10.0.128.68\\\"}],\\\"startTime\\\":\\\"2026-09-07T12:23:03Z\\\"}}\" for pod \"openshift-multus\"/\"cni-sysctl-allowlist-ds-vmtpv\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/pod?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:05.062674 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:05.062505    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:05.074399 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:05.074337    2580 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e6fa1e3b-1be4-45bc-9275-84c2d4503196\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:05Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:03Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:05Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-09-07T12:23:05Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":1,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"},\\\"containerID\\\":\\\"cri-o://ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cfde312743ce0a1d3f82fc68e67e64a3fca57853ed9c621b1fe74db36e5943ea\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"10Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-09-07T12:23:04Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0],\\\"uid\\\":0}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/ready\\\",\\\"name\\\":\\\"ready\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-pz2bv\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"10.0.128.68\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"10.0.128.68\\\"}],\\\"observedGeneration\\\":1,\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"10.0.128.68\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"10.0.128.68\\\"}],\\\"startTime\\\":\\\"2026-09-07T12:23:03Z\\\"}}\" for pod \"openshift-multus\"/\"cni-sysctl-allowlist-ds-vmtpv\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-70737425-554b-4289-aebf-e53ca32b3bc0.svc:9743/pod?timeout=10s\": dial tcp 172.30.186.39:9743: connect: connection refused" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:06.028607 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:06.028544    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv" podStartSLOduration=3.028525262 podStartE2EDuration="3.028525262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 12:23:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:23:06.027947644 +0000 UTC m=+2140.523837067" watchObservedRunningTime="2026-09-07 12:23:06.028525262 +0000 UTC m=+2140.524414692"
Sep 07 12:23:15.936499 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:15.936461    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-multus/cni-sysctl-allowlist-ds-vmtpv"]
Sep 07 12:23:15.936985 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:15.936646    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv" podUID="e6fa1e3b-1be4-45bc-9275-84c2d4503196" containerName="kube-multus-additional-cni-plugins" containerID="cri-o://ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc" gracePeriod=30
Sep 07 12:23:25.054371 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:25.054285    2580 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:25.055295 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:25.055260    2580 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:25.056098 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:25.056062    2580 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:25.056152 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:25.056112    2580 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv" podUID="e6fa1e3b-1be4-45bc-9275-84c2d4503196" containerName="kube-multus-additional-cni-plugins" probeResult="unknown"
Sep 07 12:23:35.054050 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:35.054004    2580 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:35.054951 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:35.054919    2580 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:35.055815 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:35.055787    2580 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:35.055874 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:35.055828    2580 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv" podUID="e6fa1e3b-1be4-45bc-9275-84c2d4503196" containerName="kube-multus-additional-cni-plugins" probeResult="unknown"
Sep 07 12:23:45.053892 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:45.053840    2580 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:45.054904 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:45.054874    2580 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:45.055722 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:45.055701    2580 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc" cmd=["/bin/bash","-c","test -f /ready/ready"]
Sep 07 12:23:45.055821 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:45.055736    2580 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv" podUID="e6fa1e3b-1be4-45bc-9275-84c2d4503196" containerName="kube-multus-additional-cni-plugins" probeResult="unknown"
Sep 07 12:23:46.006896 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.006869    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_cni-sysctl-allowlist-ds-vmtpv_e6fa1e3b-1be4-45bc-9275-84c2d4503196/kube-multus-additional-cni-plugins/0.log"
Sep 07 12:23:46.008999 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.008982    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:46.072639 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.072595    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ready\" (UniqueName: \"kubernetes.io/empty-dir/e6fa1e3b-1be4-45bc-9275-84c2d4503196-ready\") pod \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\" (UID: \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\") "
Sep 07 12:23:46.073099 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.072648    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pz2bv\" (UniqueName: \"kubernetes.io/projected/e6fa1e3b-1be4-45bc-9275-84c2d4503196-kube-api-access-pz2bv\") pod \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\" (UID: \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\") "
Sep 07 12:23:46.073099 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.072693    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6fa1e3b-1be4-45bc-9275-84c2d4503196-cni-sysctl-allowlist\") pod \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\" (UID: \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\") "
Sep 07 12:23:46.073099 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.072743    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6fa1e3b-1be4-45bc-9275-84c2d4503196-tuning-conf-dir\") pod \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\" (UID: \"e6fa1e3b-1be4-45bc-9275-84c2d4503196\") "
Sep 07 12:23:46.073099 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.072822    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/e6fa1e3b-1be4-45bc-9275-84c2d4503196-tuning-conf-dir" (OuterVolumeSpecName: "tuning-conf-dir") pod "e6fa1e3b-1be4-45bc-9275-84c2d4503196" (UID: "e6fa1e3b-1be4-45bc-9275-84c2d4503196"). InnerVolumeSpecName "tuning-conf-dir". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Sep 07 12:23:46.073099 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.072923    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6fa1e3b-1be4-45bc-9275-84c2d4503196-ready" (OuterVolumeSpecName: "ready") pod "e6fa1e3b-1be4-45bc-9275-84c2d4503196" (UID: "e6fa1e3b-1be4-45bc-9275-84c2d4503196"). InnerVolumeSpecName "ready". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 12:23:46.073099 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.072998    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6fa1e3b-1be4-45bc-9275-84c2d4503196-cni-sysctl-allowlist" (OuterVolumeSpecName: "cni-sysctl-allowlist") pod "e6fa1e3b-1be4-45bc-9275-84c2d4503196" (UID: "e6fa1e3b-1be4-45bc-9275-84c2d4503196"). InnerVolumeSpecName "cni-sysctl-allowlist". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 12:23:46.073356 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.073190    2580 reconciler_common.go:299] "Volume detached for volume \"ready\" (UniqueName: \"kubernetes.io/empty-dir/e6fa1e3b-1be4-45bc-9275-84c2d4503196-ready\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:23:46.073356 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.073233    2580 reconciler_common.go:299] "Volume detached for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e6fa1e3b-1be4-45bc-9275-84c2d4503196-cni-sysctl-allowlist\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:23:46.073356 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.073248    2580 reconciler_common.go:299] "Volume detached for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6fa1e3b-1be4-45bc-9275-84c2d4503196-tuning-conf-dir\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:23:46.075035 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.075015    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6fa1e3b-1be4-45bc-9275-84c2d4503196-kube-api-access-pz2bv" (OuterVolumeSpecName: "kube-api-access-pz2bv") pod "e6fa1e3b-1be4-45bc-9275-84c2d4503196" (UID: "e6fa1e3b-1be4-45bc-9275-84c2d4503196"). InnerVolumeSpecName "kube-api-access-pz2bv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 12:23:46.174043 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.174005    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pz2bv\" (UniqueName: \"kubernetes.io/projected/e6fa1e3b-1be4-45bc-9275-84c2d4503196-kube-api-access-pz2bv\") on node \"ip-10-0-128-68.ec2.internal\" DevicePath \"\""
Sep 07 12:23:46.233494 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.233403    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_cni-sysctl-allowlist-ds-vmtpv_e6fa1e3b-1be4-45bc-9275-84c2d4503196/kube-multus-additional-cni-plugins/0.log"
Sep 07 12:23:46.233494 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.233447    2580 generic.go:356] "Generic (PLEG): container finished" podID="e6fa1e3b-1be4-45bc-9275-84c2d4503196" containerID="ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc" exitCode=137
Sep 07 12:23:46.233699 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.233492    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv"
Sep 07 12:23:46.233699 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.233538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv" event={"ID":"e6fa1e3b-1be4-45bc-9275-84c2d4503196","Type":"ContainerDied","Data":"ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc"}
Sep 07 12:23:46.233699 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.233589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/cni-sysctl-allowlist-ds-vmtpv" event={"ID":"e6fa1e3b-1be4-45bc-9275-84c2d4503196","Type":"ContainerDied","Data":"4e15e1c2ab6eca780aa547c56a2c8c8c0e614a99327fcafde8768c877f928176"}
Sep 07 12:23:46.233699 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.233610    2580 scope.go:117] "RemoveContainer" containerID="ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc"
Sep 07 12:23:46.242833 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.242812    2580 scope.go:117] "RemoveContainer" containerID="ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc"
Sep 07 12:23:46.243108 ip-10-0-128-68 kubenswrapper[2580]: E0907 12:23:46.243093    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc\": container with ID starting with ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc not found: ID does not exist" containerID="ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc"
Sep 07 12:23:46.243158 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.243117    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc"} err="failed to get container status \"ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc\": rpc error: code = NotFound desc = could not find container \"ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc\": container with ID starting with ed41c3eae167e7b2c9aaa5a16f75c9b704e42364179c833147089c1fcf2b14fc not found: ID does not exist"
Sep 07 12:23:46.253772 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.253731    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-multus/cni-sysctl-allowlist-ds-vmtpv"]
Sep 07 12:23:46.255206 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:46.255174    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-multus/cni-sysctl-allowlist-ds-vmtpv"]
Sep 07 12:23:48.010485 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:23:48.010449    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6fa1e3b-1be4-45bc-9275-84c2d4503196" path="/var/lib/kubelet/pods/e6fa1e3b-1be4-45bc-9275-84c2d4503196/volumes"
Sep 07 12:25:26.526911 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:25:26.526824    2580 scope.go:117] "RemoveContainer" containerID="aeeeee80c9b6bad292bce5aac580a8e18684ec5c45b975f0c9722c41796b4c2f"
Sep 07 12:25:26.535103 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:25:26.535081    2580 scope.go:117] "RemoveContainer" containerID="68f944ce750bf9320ce661e0f2d13dc91b6e0481e605ba9cd5fbc51f2887d803"
Sep 07 12:27:26.570166 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:26.570129    2580 scope.go:117] "RemoveContainer" containerID="226cd126fec52b2ce476c270b95c6efaabad9d2adf8594560f225cd668c4a49d"
Sep 07 12:27:26.579475 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:26.579456    2580 scope.go:117] "RemoveContainer" containerID="c310d420be99da50707c5406733cdf44c07e753ef2b508b8e952cc9743d91bba"
Sep 07 12:27:47.158296 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.158256    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59894548fc-65nd9"]
Sep 07 12:27:47.158670 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.158580    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerName="istio-proxy"
Sep 07 12:27:47.158670 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.158591    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="554cfa21-b6f6-44e2-9d05-c46047bb3539" containerName="istio-proxy"
Sep 07 12:27:47.158670 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.158601    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6fa1e3b-1be4-45bc-9275-84c2d4503196" containerName="kube-multus-additional-cni-plugins"
Sep 07 12:27:47.158670 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.158606    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6fa1e3b-1be4-45bc-9275-84c2d4503196" containerName="kube-multus-additional-cni-plugins"
Sep 07 12:27:47.158670 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.158664    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d52eecbf-1cd0-44ec-822b-b31cc579adbb" containerName="istio-proxy"
Sep 07 12:27:47.158670 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.158672    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6fa1e3b-1be4-45bc-9275-84c2d4503196" containerName="kube-multus-additional-cni-plugins"
Sep 07 12:27:47.161587 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.161564    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9"
Sep 07 12:27:47.168296 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.168264    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 07 12:27:47.169689 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.169657    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 07 12:27:47.169853 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.169753    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 12:27:47.170253 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.170179    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad16c99-0196-4581-82de-df2d4c2fed4f-metrics-certs\") pod \"maas-api-59894548fc-65nd9\" (UID: \"5ad16c99-0196-4581-82de-df2d4c2fed4f\") " pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9"
Sep 07 12:27:47.170500 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.170472    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad16c99-0196-4581-82de-df2d4c2fed4f-openshift-service-ca\") pod \"maas-api-59894548fc-65nd9\" (UID: \"5ad16c99-0196-4581-82de-df2d4c2fed4f\") " pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9"
Sep 07 12:27:47.170664 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.170543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm558\" (UniqueName: \"kubernetes.io/projected/5ad16c99-0196-4581-82de-df2d4c2fed4f-kube-api-access-vm558\") pod \"maas-api-59894548fc-65nd9\" (UID: \"5ad16c99-0196-4581-82de-df2d4c2fed4f\") " pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9"
Sep 07 12:27:47.170664 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.170615    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad16c99-0196-4581-82de-df2d4c2fed4f-maas-api-tls\") pod \"maas-api-59894548fc-65nd9\" (UID: \"5ad16c99-0196-4581-82de-df2d4c2fed4f\") " pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9"
Sep 07 12:27:47.170839 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.170752    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 07 12:27:47.171123 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.171102    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p4vz9\""
Sep 07 12:27:47.172956 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.172934    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59894548fc-65nd9"]
Sep 07 12:27:47.271110 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.271078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad16c99-0196-4581-82de-df2d4c2fed4f-maas-api-tls\") pod \"maas-api-59894548fc-65nd9\" (UID: \"5ad16c99-0196-4581-82de-df2d4c2fed4f\") " pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9"
Sep 07 12:27:47.271354 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.271141    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad16c99-0196-4581-82de-df2d4c2fed4f-metrics-certs\") pod \"maas-api-59894548fc-65nd9\" (UID: \"5ad16c99-0196-4581-82de-df2d4c2fed4f\") " pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9"
Sep 07 12:27:47.271354 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.271186    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad16c99-0196-4581-82de-df2d4c2fed4f-openshift-service-ca\") pod \"maas-api-59894548fc-65nd9\" (UID: \"5ad16c99-0196-4581-82de-df2d4c2fed4f\") " pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9"
Sep 07 12:27:47.271354 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.271255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm558\" (UniqueName: \"kubernetes.io/projected/5ad16c99-0196-4581-82de-df2d4c2fed4f-kube-api-access-vm558\") pod \"maas-api-59894548fc-65nd9\" (UID: \"5ad16c99-0196-4581-82de-df2d4c2fed4f\") " pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9"
Sep 07 12:27:47.271908 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.271883    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad16c99-0196-4581-82de-df2d4c2fed4f-openshift-service-ca\") pod \"maas-api-59894548fc-65nd9\" (UID: \"5ad16c99-0196-4581-82de-df2d4c2fed4f\") " pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9"
Sep 07 12:27:47.273597 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.273555    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ad16c99-0196-4581-82de-df2d4c2fed4f-metrics-certs\") pod \"maas-api-59894548fc-65nd9\" (UID: \"5ad16c99-0196-4581-82de-df2d4c2fed4f\") " pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9"
Sep 07 12:27:47.273744 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.273672    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ad16c99-0196-4581-82de-df2d4c2fed4f-maas-api-tls\") pod \"maas-api-59894548fc-65nd9\" (UID: \"5ad16c99-0196-4581-82de-df2d4c2fed4f\") " pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9"
Sep 07 12:27:47.278574 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.278552    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm558\" (UniqueName: \"kubernetes.io/projected/5ad16c99-0196-4581-82de-df2d4c2fed4f-kube-api-access-vm558\") pod \"maas-api-59894548fc-65nd9\" (UID: \"5ad16c99-0196-4581-82de-df2d4c2fed4f\") " pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9"
Sep 07 12:27:47.480497 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.480403    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9"
Sep 07 12:27:47.753757 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.753725    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59894548fc-65nd9"]
Sep 07 12:27:47.756365 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:27:47.756329    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad16c99_0196_4581_82de_df2d4c2fed4f.slice/crio-41975226bb5225a86442b997d3b7e18d612a9b480beffbda8554c50ed0c8b231 WatchSource:0}: Error finding container 41975226bb5225a86442b997d3b7e18d612a9b480beffbda8554c50ed0c8b231: Status 404 returned error can't find the container with id 41975226bb5225a86442b997d3b7e18d612a9b480beffbda8554c50ed0c8b231
Sep 07 12:27:47.757691 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:47.757675    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 12:27:48.314105 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:48.314066    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9" event={"ID":"5ad16c99-0196-4581-82de-df2d4c2fed4f","Type":"ContainerStarted","Data":"41975226bb5225a86442b997d3b7e18d612a9b480beffbda8554c50ed0c8b231"}
Sep 07 12:27:50.324612 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:50.324577    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9" event={"ID":"5ad16c99-0196-4581-82de-df2d4c2fed4f","Type":"ContainerStarted","Data":"373ba6fcfbdf95d72f1ecc61b273c167ead34af2651b035d906d56d3aa2b7f0f"}
Sep 07 12:27:50.325008 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:50.324707    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9"
Sep 07 12:27:50.342352 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:50.342267    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9" podStartSLOduration=1.748764128 podStartE2EDuration="3.342250207s" totalImagesPullingTime="1.593486079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 12:27:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:27:50.341813746 +0000 UTC m=+2424.837703169" watchObservedRunningTime="2026-09-07 12:27:50.342250207 +0000 UTC m=+2424.838139629"
Sep 07 12:27:56.334565 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:27:56.334535    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59894548fc-65nd9"
Sep 07 12:28:17.194072 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:17.194043    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-648db6df7b-2r8zx_77d29c3a-d28b-4d5b-8aec-29d25c6750af/copy-manifests/0.log"
Sep 07 12:28:17.212734 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:17.212698    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-648db6df7b-2r8zx_77d29c3a-d28b-4d5b-8aec-29d25c6750af/manager/0.log"
Sep 07 12:28:17.753837 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:17.753805    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-59894548fc-65nd9_5ad16c99-0196-4581-82de-df2d4c2fed4f/maas-api/0.log"
Sep 07 12:28:19.851132 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:19.851098    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff_7a3f497d-1b05-434f-a4b4-69dc09a87643/istio-proxy/0.log"
Sep 07 12:28:19.972233 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:19.972189    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-x5jvb_83e0ccb3-22c8-478a-9635-1928e6bcc2c7/discovery/0.log"
Sep 07 12:28:20.303763 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:20.303733    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6pgwj_5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952/istio-proxy/1.log"
Sep 07 12:28:20.534492 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:20.534454    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-5kjj5_8eaded52-ec7d-4d87-af2f-cd8e7cfe2049/payload-processing/0.log"
Sep 07 12:28:21.085475 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:21.085445    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-4drnc_6cc84c3c-5818-470f-8404-5492da0d92d6/main/0.log"
Sep 07 12:28:21.304372 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:21.304334    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-gx45m_6a97ca01-ebfc-4b5c-a963-95d99142d527/main/0.log"
Sep 07 12:28:29.935527 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:29.935490    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8s4jc_ec37e993-169b-4dc5-b5f6-e938f46e33d7/global-pull-secret-syncer/0.log"
Sep 07 12:28:30.090935 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:30.090905    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hwhqc_8a42548a-1c35-4dc8-8ddd-8d6777ac544e/konnectivity-agent/0.log"
Sep 07 12:28:30.147439 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:30.147411    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-68.ec2.internal_433f8b6f5fd9ac28e74f186701608de8/haproxy/0.log"
Sep 07 12:28:37.146796 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:37.146763    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cfdtb_f2fc51c1-3f3a-4760-93b6-64fa09cedaa3/node-exporter/0.log"
Sep 07 12:28:37.164894 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:37.164863    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cfdtb_f2fc51c1-3f3a-4760-93b6-64fa09cedaa3/kube-rbac-proxy/0.log"
Sep 07 12:28:37.185294 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:37.185273    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cfdtb_f2fc51c1-3f3a-4760-93b6-64fa09cedaa3/init-textfile/0.log"
Sep 07 12:28:37.326673 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:37.326646    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-sfl7k_aa44cf6c-3099-4514-aa93-28ee27dbf199/kube-rbac-proxy-main/0.log"
Sep 07 12:28:37.355069 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:37.355046    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-sfl7k_aa44cf6c-3099-4514-aa93-28ee27dbf199/kube-rbac-proxy-self/0.log"
Sep 07 12:28:37.373260 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:37.373222    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-sfl7k_aa44cf6c-3099-4514-aa93-28ee27dbf199/openshift-state-metrics/0.log"
Sep 07 12:28:37.430596 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:37.430509    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_30122c97-d13e-40c9-8dff-d6ef59cc6edd/prometheus/0.log"
Sep 07 12:28:37.451249 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:37.451224    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_30122c97-d13e-40c9-8dff-d6ef59cc6edd/config-reloader/0.log"
Sep 07 12:28:37.467804 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:37.467779    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_30122c97-d13e-40c9-8dff-d6ef59cc6edd/thanos-sidecar/0.log"
Sep 07 12:28:37.486323 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:37.486299    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_30122c97-d13e-40c9-8dff-d6ef59cc6edd/kube-rbac-proxy-web/0.log"
Sep 07 12:28:37.506791 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:37.506771    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_30122c97-d13e-40c9-8dff-d6ef59cc6edd/kube-rbac-proxy/0.log"
Sep 07 12:28:37.528598 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:37.528576    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_30122c97-d13e-40c9-8dff-d6ef59cc6edd/kube-rbac-proxy-thanos/0.log"
Sep 07 12:28:37.547485 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:37.547460    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_30122c97-d13e-40c9-8dff-d6ef59cc6edd/init-config-reloader/0.log"
Sep 07 12:28:40.435503 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.435466    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"]
Sep 07 12:28:40.438823 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.438800    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:40.441307 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.441277    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bzv24\"/\"openshift-service-ca.crt\""
Sep 07 12:28:40.442281 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.442259    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bzv24\"/\"kube-root-ca.crt\""
Sep 07 12:28:40.442395 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.442278    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bzv24\"/\"default-dockercfg-fprf9\""
Sep 07 12:28:40.450274 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.450250    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"]
Sep 07 12:28:40.551166 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.551130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4e876c45-4877-4fb9-a950-665d7df22505-proc\") pod \"perf-node-gather-daemonset-tdhjd\" (UID: \"4e876c45-4877-4fb9-a950-665d7df22505\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:40.551396 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.551184    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvth4\" (UniqueName: \"kubernetes.io/projected/4e876c45-4877-4fb9-a950-665d7df22505-kube-api-access-hvth4\") pod \"perf-node-gather-daemonset-tdhjd\" (UID: \"4e876c45-4877-4fb9-a950-665d7df22505\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:40.551396 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.551267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4e876c45-4877-4fb9-a950-665d7df22505-podres\") pod \"perf-node-gather-daemonset-tdhjd\" (UID: \"4e876c45-4877-4fb9-a950-665d7df22505\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:40.551396 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.551289    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e876c45-4877-4fb9-a950-665d7df22505-sys\") pod \"perf-node-gather-daemonset-tdhjd\" (UID: \"4e876c45-4877-4fb9-a950-665d7df22505\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:40.551396 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.551314    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e876c45-4877-4fb9-a950-665d7df22505-lib-modules\") pod \"perf-node-gather-daemonset-tdhjd\" (UID: \"4e876c45-4877-4fb9-a950-665d7df22505\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:40.652635 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.652602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4e876c45-4877-4fb9-a950-665d7df22505-podres\") pod \"perf-node-gather-daemonset-tdhjd\" (UID: \"4e876c45-4877-4fb9-a950-665d7df22505\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:40.652635 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.652640    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e876c45-4877-4fb9-a950-665d7df22505-sys\") pod \"perf-node-gather-daemonset-tdhjd\" (UID: \"4e876c45-4877-4fb9-a950-665d7df22505\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:40.652881 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.652661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e876c45-4877-4fb9-a950-665d7df22505-lib-modules\") pod \"perf-node-gather-daemonset-tdhjd\" (UID: \"4e876c45-4877-4fb9-a950-665d7df22505\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:40.652881 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.652718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4e876c45-4877-4fb9-a950-665d7df22505-proc\") pod \"perf-node-gather-daemonset-tdhjd\" (UID: \"4e876c45-4877-4fb9-a950-665d7df22505\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:40.652881 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.652748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvth4\" (UniqueName: \"kubernetes.io/projected/4e876c45-4877-4fb9-a950-665d7df22505-kube-api-access-hvth4\") pod \"perf-node-gather-daemonset-tdhjd\" (UID: \"4e876c45-4877-4fb9-a950-665d7df22505\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:40.652881 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.652753    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e876c45-4877-4fb9-a950-665d7df22505-sys\") pod \"perf-node-gather-daemonset-tdhjd\" (UID: \"4e876c45-4877-4fb9-a950-665d7df22505\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:40.652881 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.652784    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4e876c45-4877-4fb9-a950-665d7df22505-podres\") pod \"perf-node-gather-daemonset-tdhjd\" (UID: \"4e876c45-4877-4fb9-a950-665d7df22505\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:40.652881 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.652825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4e876c45-4877-4fb9-a950-665d7df22505-proc\") pod \"perf-node-gather-daemonset-tdhjd\" (UID: \"4e876c45-4877-4fb9-a950-665d7df22505\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:40.652881 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.652837    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e876c45-4877-4fb9-a950-665d7df22505-lib-modules\") pod \"perf-node-gather-daemonset-tdhjd\" (UID: \"4e876c45-4877-4fb9-a950-665d7df22505\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:40.659921 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.659903    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvth4\" (UniqueName: \"kubernetes.io/projected/4e876c45-4877-4fb9-a950-665d7df22505-kube-api-access-hvth4\") pod \"perf-node-gather-daemonset-tdhjd\" (UID: \"4e876c45-4877-4fb9-a950-665d7df22505\") " pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:40.749110 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:40.749025    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:41.019277 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:41.019185    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"]
Sep 07 12:28:41.022405 ip-10-0-128-68 kubenswrapper[2580]: W0907 12:28:41.022377    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4e876c45_4877_4fb9_a950_665d7df22505.slice/crio-d67eb889fa02e7c327861140cc5845500958e37bb4a4383ffd66298dbc3af16b WatchSource:0}: Error finding container d67eb889fa02e7c327861140cc5845500958e37bb4a4383ffd66298dbc3af16b: Status 404 returned error can't find the container with id d67eb889fa02e7c327861140cc5845500958e37bb4a4383ffd66298dbc3af16b
Sep 07 12:28:41.552181 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:41.552144    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd" event={"ID":"4e876c45-4877-4fb9-a950-665d7df22505","Type":"ContainerStarted","Data":"2428fdb911e8c8e59a65785f88dd62582a473867e939da66e54fdab560dac157"}
Sep 07 12:28:41.552574 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:41.552215    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd" event={"ID":"4e876c45-4877-4fb9-a950-665d7df22505","Type":"ContainerStarted","Data":"d67eb889fa02e7c327861140cc5845500958e37bb4a4383ffd66298dbc3af16b"}
Sep 07 12:28:41.552574 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:41.552237    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:41.567717 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:41.567666    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd" podStartSLOduration=1.567648245 podStartE2EDuration="1.567648245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 12:28:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 12:28:41.566173202 +0000 UTC m=+2476.062062622" watchObservedRunningTime="2026-09-07 12:28:41.567648245 +0000 UTC m=+2476.063537668"
Sep 07 12:28:42.475006 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:42.474970    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jsx9s_17691127-7e46-44e2-a052-edf0873d6f26/dns/0.log"
Sep 07 12:28:42.493081 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:42.493057    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jsx9s_17691127-7e46-44e2-a052-edf0873d6f26/kube-rbac-proxy/0.log"
Sep 07 12:28:42.577279 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:42.577240    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kwr4n_3c2e42f2-bf8f-4093-b496-af2041e0090e/dns-node-resolver/0.log"
Sep 07 12:28:43.187344 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:43.187311    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9f27p_8722b24c-1b27-4443-aacb-b96b30a04bf3/node-ca/0.log"
Sep 07 12:28:44.200980 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:44.200954    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff_7a3f497d-1b05-434f-a4b4-69dc09a87643/istio-proxy/0.log"
Sep 07 12:28:44.753503 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:44.753466    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-x5jvb_83e0ccb3-22c8-478a-9635-1928e6bcc2c7/discovery/0.log"
Sep 07 12:28:44.827805 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:44.827767    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6pgwj_5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952/istio-proxy/0.log"
Sep 07 12:28:44.846281 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:44.846248    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6pgwj_5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952/istio-proxy/1.log"
Sep 07 12:28:44.929895 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:44.929861    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-5kjj5_8eaded52-ec7d-4d87-af2f-cd8e7cfe2049/payload-processing/0.log"
Sep 07 12:28:45.913224 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:45.913170    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vb6cl_02120513-1881-4f03-89fb-c94109d599b4/serve-healthcheck-canary/0.log"
Sep 07 12:28:46.578589 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:46.578554    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zcbwr_6f7985a9-98c1-450f-9e35-5401a6574918/kube-rbac-proxy/0.log"
Sep 07 12:28:46.597858 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:46.597829    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zcbwr_6f7985a9-98c1-450f-9e35-5401a6574918/exporter/0.log"
Sep 07 12:28:46.618173 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:46.618143    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zcbwr_6f7985a9-98c1-450f-9e35-5401a6574918/extractor/0.log"
Sep 07 12:28:47.565138 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:47.565109    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bzv24/perf-node-gather-daemonset-tdhjd"
Sep 07 12:28:49.279847 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:49.279815    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-648db6df7b-2r8zx_77d29c3a-d28b-4d5b-8aec-29d25c6750af/manager/0.log"
Sep 07 12:28:49.295783 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:49.295756    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-648db6df7b-2r8zx_77d29c3a-d28b-4d5b-8aec-29d25c6750af/copy-manifests/0.log"
Sep 07 12:28:50.893904 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:50.893871    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59499dc5ff-vc8vh_d28abc0d-55e9-49b2-8396-059e2a514373/manager/0.log"
Sep 07 12:28:50.903095 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:50.903060    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff_7a3f497d-1b05-434f-a4b4-69dc09a87643/istio-proxy/0.log"
Sep 07 12:28:51.449839 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:51.449803    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-x5jvb_83e0ccb3-22c8-478a-9635-1928e6bcc2c7/discovery/0.log"
Sep 07 12:28:51.496616 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:51.496581    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6pgwj_5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952/istio-proxy/0.log"
Sep 07 12:28:51.514574 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:51.514546    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6pgwj_5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952/istio-proxy/1.log"
Sep 07 12:28:51.582870 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:51.582834    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-5kjj5_8eaded52-ec7d-4d87-af2f-cd8e7cfe2049/payload-processing/0.log"
Sep 07 12:28:53.106069 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:53.106040    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-2xrff_7a3f497d-1b05-434f-a4b4-69dc09a87643/istio-proxy/0.log"
Sep 07 12:28:53.674216 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:53.674170    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-x5jvb_83e0ccb3-22c8-478a-9635-1928e6bcc2c7/discovery/0.log"
Sep 07 12:28:53.724184 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:53.724150    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6pgwj_5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952/istio-proxy/0.log"
Sep 07 12:28:53.741488 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:53.741454    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6pgwj_5f7b6385-d2f7-46a9-a1be-3f8dbf8ff952/istio-proxy/1.log"
Sep 07 12:28:53.878302 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:53.878261    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-5kjj5_8eaded52-ec7d-4d87-af2f-cd8e7cfe2049/payload-processing/0.log"
Sep 07 12:28:57.465492 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:57.465467    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4fj7g_a8041043-f46e-4a5f-8e1f-01018a5189bb/kube-multus-additional-cni-plugins/0.log"
Sep 07 12:28:57.485109 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:57.485064    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4fj7g_a8041043-f46e-4a5f-8e1f-01018a5189bb/egress-router-binary-copy/0.log"
Sep 07 12:28:57.509014 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:57.508982    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4fj7g_a8041043-f46e-4a5f-8e1f-01018a5189bb/cni-plugins/0.log"
Sep 07 12:28:57.527148 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:57.527123    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4fj7g_a8041043-f46e-4a5f-8e1f-01018a5189bb/bond-cni-plugin/0.log"
Sep 07 12:28:57.545127 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:57.545100    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4fj7g_a8041043-f46e-4a5f-8e1f-01018a5189bb/routeoverride-cni/0.log"
Sep 07 12:28:57.563212 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:57.563170    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4fj7g_a8041043-f46e-4a5f-8e1f-01018a5189bb/whereabouts-cni-bincopy/0.log"
Sep 07 12:28:57.581673 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:57.581648    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4fj7g_a8041043-f46e-4a5f-8e1f-01018a5189bb/whereabouts-cni/0.log"
Sep 07 12:28:57.968737 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:57.968700    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mq4kn_7300402b-f9fd-4f08-8728-b3a4ad8e2d8e/kube-multus/0.log"
Sep 07 12:28:58.091138 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:58.091106    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4dgj8_7e4e91b4-23e4-4903-8e2d-cf1d2145b51f/network-metrics-daemon/0.log"
Sep 07 12:28:58.105807 ip-10-0-128-68 kubenswrapper[2580]: I0907 12:28:58.105777    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4dgj8_7e4e91b4-23e4-4903-8e2d-cf1d2145b51f/kube-rbac-proxy/0.log"