Sep 14 02:52:17.106452 ip-10-0-138-244 systemd[1]: Starting Kubernetes Kubelet...
Sep 14 02:52:17.532189 ip-10-0-138-244 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 02:52:17.532189 ip-10-0-138-244 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 14 02:52:17.532189 ip-10-0-138-244 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 02:52:17.532189 ip-10-0-138-244 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 14 02:52:17.532189 ip-10-0-138-244 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 02:52:17.534086 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.534007    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 14 02:52:17.539556 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539535    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:52:17.539556 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539552    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:52:17.539556 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539556    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:52:17.539556 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539559    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:52:17.539556 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539562    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539566    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539569    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539572    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539575    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539577    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539580    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539582    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539585    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539588    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539590    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539594    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539598    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539601    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539604    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539607    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539610    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539613    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539616    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539618    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:52:17.539745 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539621    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539624    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539626    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539628    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539631    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539634    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539636    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539639    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539642    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539645    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539648    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539650    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539653    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539655    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539659    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539662    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539664    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539668    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539671    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:52:17.540238 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539675    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539677    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539680    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539682    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539685    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539688    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539691    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539694    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539696    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539699    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539701    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539704    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539706    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539709    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539711    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539714    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539717    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539719    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539722    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539724    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:52:17.540692 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539726    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539729    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539731    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539734    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539736    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539740    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539742    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539746    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539749    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539752    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539755    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539757    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539760    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539763    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539765    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539768    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539770    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539773    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539775    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:52:17.541194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539777    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539781    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539784    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539786    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539789    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539791    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539794    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539796    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539799    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539801    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539804    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539806    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539810    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539814    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539817    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539820    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.539823    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540523    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540530    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:52:17.541705 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540533    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540537    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540542    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540545    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540547    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540550    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540554    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540557    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540561    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540564    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540567    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540569    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540572    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540574    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540577    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540580    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540583    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540585    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540588    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:52:17.542167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540590    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540593    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540595    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540598    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540600    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540602    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540605    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540607    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540610    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540612    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540615    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540617    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540620    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540623    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540626    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540628    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540631    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540634    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540636    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:52:17.542740 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540639    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540641    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540644    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540646    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540649    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540651    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540654    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540656    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540659    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540661    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540664    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540667    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540670    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540672    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540675    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540677    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540680    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540683    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540685    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:52:17.543229 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540688    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540692    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540695    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540698    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540701    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540703    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540706    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540709    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540712    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540715    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540718    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540720    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540723    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540725    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540727    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540730    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540732    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540735    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540737    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:52:17.543701 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540741    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540743    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540746    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540748    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540751    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540754    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540757    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540759    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540762    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540764    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540767    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540769    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540772    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540774    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540777    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540779    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540781    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540784    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540787    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540790    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:52:17.544167 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.540792    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541143    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541148    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541151    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541153    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541156    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541158    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541161    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541164    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541167    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541189    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541195    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541199    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541203    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541206    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541209    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541211    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541215    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541218    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541220    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:52:17.544661 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541223    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541225    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541228    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541230    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541233    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541235    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541238    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541240    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541243    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541245    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541248    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541251    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541254    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541257    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541259    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541262    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541265    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541268    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541272    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:52:17.545165 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541276    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541279    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541282    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541284    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541287    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541290    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541292    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541295    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541297    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541300    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541302    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541305    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541307    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541310    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541312    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541315    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541317    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541320    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541322    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541325    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:52:17.545664 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541328    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541330    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541333    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541335    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541338    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541340    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541343    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541345    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541348    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541350    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541353    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541355    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541358    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541360    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541363    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541365    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541368    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541378    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541381    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541383    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:52:17.546189 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541386    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541388    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541391    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541394    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541396    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541399    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541402    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541405    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541407    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541410    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541413    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541417    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541421    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541424    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541426    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541429    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541432    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541435    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541438    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:52:17.546718 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541441    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.541443    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541518    2579 flags.go:64] FLAG: --address="0.0.0.0"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541549    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541555    2579 flags.go:64] FLAG: --anonymous-auth="true"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541559    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541563    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541566    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541570    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541574    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541577    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541579    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541583    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541586    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541589    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541592    2579 flags.go:64] FLAG: --cgroup-root=""
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541594    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541597    2579 flags.go:64] FLAG: --client-ca-file=""
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541600    2579 flags.go:64] FLAG: --cloud-provider="external"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541603    2579 flags.go:64] FLAG: --cluster-dns="[]"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541607    2579 flags.go:64] FLAG: --cluster-domain=""
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541610    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541613    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541616    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 14 02:52:17.547191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541619    2579 flags.go:64] FLAG: --container-log-max-files="5"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541623    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541626    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541629    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541632    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541643    2579 flags.go:64] FLAG: --contention-profiling="false"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541646    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541649    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541653    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541656    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541660    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541663    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541665    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541668    2579 flags.go:64] FLAG: --enable-load-reader="false"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541671    2579 flags.go:64] FLAG: --enable-server="true"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541674    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541678    2579 flags.go:64] FLAG: --event-burst="100"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541681    2579 flags.go:64] FLAG: --event-qps="50"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541684    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541686    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541689    2579 flags.go:64] FLAG: --eviction-hard=""
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541697    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541700    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541703    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541706    2579 flags.go:64] FLAG: --eviction-soft=""
Sep 14 02:52:17.547847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541709    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541712    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541716    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541719    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541721    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541724    2579 flags.go:64] FLAG: --fail-swap-on="true"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541727    2579 flags.go:64] FLAG: --feature-gates=""
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541730    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541733    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541736    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541739    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541742    2579 flags.go:64] FLAG: --healthz-port="10248"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541745    2579 flags.go:64] FLAG: --help="false"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541748    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-138-244.ec2.internal"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541751    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541754    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541757    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541760    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541764    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541766    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541769    2579 flags.go:64] FLAG: --image-service-endpoint=""
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541772    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541774    2579 flags.go:64] FLAG: --kube-api-burst="100"
Sep 14 02:52:17.548459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541777    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541780    2579 flags.go:64] FLAG: --kube-api-qps="50"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541783    2579 flags.go:64] FLAG: --kube-reserved=""
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541785    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541788    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541791    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541794    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541796    2579 flags.go:64] FLAG: --lock-file=""
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541799    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541802    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541805    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541810    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541813    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541816    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541818    2579 flags.go:64] FLAG: --logging-format="text"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541821    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541824    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541827    2579 flags.go:64] FLAG: --manifest-url=""
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541829    2579 flags.go:64] FLAG: --manifest-url-header=""
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541833    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541836    2579 flags.go:64] FLAG: --max-open-files="1000000"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541840    2579 flags.go:64] FLAG: --max-pods="110"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541842    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541845    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541848    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 14 02:52:17.549055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541851    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541853    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541856    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541859    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541866    2579 flags.go:64] FLAG: --node-status-max-images="50"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541869    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541872    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541874    2579 flags.go:64] FLAG: --pod-cidr=""
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541877    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541882    2579 flags.go:64] FLAG: --pod-manifest-path=""
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541884    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541887    2579 flags.go:64] FLAG: --pods-per-core="0"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541890    2579 flags.go:64] FLAG: --port="10250"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541893    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541895    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02501501eb4544980"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541898    2579 flags.go:64] FLAG: --qos-reserved=""
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541901    2579 flags.go:64] FLAG: --read-only-port="10255"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541904    2579 flags.go:64] FLAG: --register-node="true"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541909    2579 flags.go:64] FLAG: --register-with-taints=""
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541912    2579 flags.go:64] FLAG: --registry-burst="10"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541915    2579 flags.go:64] FLAG: --registry-qps="5"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541918    2579 flags.go:64] FLAG: --reserved-cpus=""
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541921    2579 flags.go:64] FLAG: --reserved-memory=""
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541924    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541927    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 14 02:52:17.549704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541930    2579 flags.go:64] FLAG: --rotate-certificates="false"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541933    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541936    2579 flags.go:64] FLAG: --runonce="false"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541938    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541941    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541946    2579 flags.go:64] FLAG: --seccomp-default="false"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541949    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541951    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541954    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541957    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541960    2579 flags.go:64] FLAG: --storage-driver-password="root"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541963    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541966    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541968    2579 flags.go:64] FLAG: --storage-driver-user="root"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541971    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541974    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541977    2579 flags.go:64] FLAG: --system-cgroups=""
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541980    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541985    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541987    2579 flags.go:64] FLAG: --tls-cert-file=""
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541990    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541994    2579 flags.go:64] FLAG: --tls-min-version=""
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541997    2579 flags.go:64] FLAG: --tls-private-key-file=""
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.541999    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.542002    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 14 02:52:17.550323 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.542006    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.542009    2579 flags.go:64] FLAG: --v="2"
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.542013    2579 flags.go:64] FLAG: --version="false"
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.542017    2579 flags.go:64] FLAG: --vmodule=""
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.542021    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.542024    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542131    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542135    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542138    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542141    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542143    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542146    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542150    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542153    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542156    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542159    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542163    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542166    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542184    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542187    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:52:17.550943 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542190    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542193    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542196    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542199    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542201    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542204    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542208    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542212    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542215    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542218    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542221    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542223    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542228    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542231    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542234    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542237    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542241    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542243    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542246    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:52:17.551469 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542248    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542251    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542253    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542256    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542258    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542262    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542265    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542267    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542270    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542273    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542275    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542278    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542280    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542283    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542285    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542288    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542291    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542293    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542296    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542298    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:52:17.551940 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542301    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542304    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542306    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542309    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542311    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542316    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542318    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542322    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542325    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542328    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542331    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542334    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542337    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542339    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542342    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542344    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542347    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542351    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542354    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542356    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:52:17.552441 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542359    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542361    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542364    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542367    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542370    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542372    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542375    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542378    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542380    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542382    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542385    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542387    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542390    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542393    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542395    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542398    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542400    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542404    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542407    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542409    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:52:17.552922 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542412    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:52:17.553427 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542415    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:52:17.553427 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542417    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:52:17.553427 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542420    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:52:17.553427 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542423    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:52:17.553427 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.542425    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:52:17.553427 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.542431    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 02:52:17.553427 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.550275    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 14 02:52:17.553427 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.550294    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 14 02:52:17.553427 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.551227    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 14 02:52:17.553427 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.551235    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 14 02:52:17.553427 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551290    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:52:17.553427 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551295    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:52:17.553427 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551298    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:52:17.553427 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551301    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551304    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551307    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551310    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551313    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551315    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551318    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551321    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551323    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551326    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551329    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551332    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551335    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551337    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551340    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551342    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551345    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551348    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551350    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551354    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:52:17.553791 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551356    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551359    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551361    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551364    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551366    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551369    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551373    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551375    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551378    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551380    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551383    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551385    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551388    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551390    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551393    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551395    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551398    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551400    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551403    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:52:17.554285 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551405    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551408    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551410    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551413    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551415    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551418    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551420    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551423    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551425    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551428    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551430    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551433    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551435    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551438    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551441    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551443    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551446    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551448    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551451    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551454    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:52:17.554789 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551459    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551463    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551466    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551469    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551471    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551474    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551477    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551479    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551482    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551486    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551489    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551493    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551496    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551498    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551501    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551505    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551508    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551511    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551514    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:52:17.555287 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551517    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551520    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551523    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551526    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551529    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551531    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551534    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551537    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551539    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551542    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551545    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551548    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551551    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551554    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551557    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551560    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551563    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:52:17.555752 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551565    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:52:17.556155 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.551570    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 02:52:17.556155 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551675    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 02:52:17.556155 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551679    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 02:52:17.556155 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551681    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 02:52:17.556155 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551684    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 02:52:17.556155 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551687    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 02:52:17.556155 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551690    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 02:52:17.556155 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551693    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 02:52:17.556155 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551695    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 02:52:17.556155 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551699    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 02:52:17.556155 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551703    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 02:52:17.556155 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551706    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 02:52:17.556155 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551709    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 02:52:17.556155 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551711    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 02:52:17.556155 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551714    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551717    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551720    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551722    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551725    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551727    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551730    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551732    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551734    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551737    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551739    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551742    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551744    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551747    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551750    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551753    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551755    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551758    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551760    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 02:52:17.556533 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551763    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551765    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551768    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551770    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551772    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551775    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551777    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551780    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551782    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551785    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551787    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551790    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551792    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551795    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551797    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551800    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551802    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551804    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551807    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 02:52:17.556984 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551809    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551813    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551816    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551819    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551822    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551824    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551828    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551831    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551834    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551837    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551839    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551842    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551845    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551847    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551850    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551853    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551855    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551858    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551860    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551862    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 02:52:17.557465 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551865    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551868    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551870    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551873    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551875    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551878    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551880    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551883    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551885    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551887    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551890    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551893    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551895    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551898    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551900    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551903    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551905    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551908    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551910    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551913    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 02:52:17.557957 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551916    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 02:52:17.558540 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551918    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 02:52:17.558540 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551921    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 02:52:17.558540 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551923    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 02:52:17.558540 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551926    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 02:52:17.558540 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551928    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 02:52:17.558540 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551931    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 02:52:17.558540 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:17.551935    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 02:52:17.558540 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.551940    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 02:52:17.558540 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.552030    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 14 02:52:17.558540 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.554324    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 14 02:52:17.558540 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.555396    2579 server.go:1019] "Starting client certificate rotation"
Sep 14 02:52:17.558540 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.555491    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 02:52:17.558540 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.555524    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 02:52:17.558859 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.558342    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 02:52:17.561284 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.561256    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 02:52:17.575229 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.575209    2579 log.go:25] "Validated CRI v1 runtime API"
Sep 14 02:52:17.585991 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.585975    2579 log.go:25] "Validated CRI v1 image API"
Sep 14 02:52:17.586814 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.586800    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 14 02:52:17.591189 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.591149    2579 fs.go:135] Filesystem UUIDs: map[50a76716-78dd-4c22-9e87-6a73c30311a1:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 89c486b4-ef39-43c0-9890-49704ddc1031:/dev/nvme0n1p4]
Sep 14 02:52:17.591281 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.591188    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 14 02:52:17.591368 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.591350    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 02:52:17.596596 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.596476    2579 manager.go:216] Machine: {Timestamp:2026-09-14 02:52:17.595128821 +0000 UTC m=+0.381190725 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3215868 MemoryCapacity:32812142592 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20c596b7d020fec41fee5202795078 SystemUUID:ec20c596-b7d0-20fe-c41f-ee5202795078 BootID:7e2d7e55-d992-4e88-a0ee-a9dd4c9e61f7 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406073344 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406069248 Type:vfs Inodes:4005388 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e6:a4:0c:bd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e6:a4:0c:bd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3e:56:fd:4a:a5:ea Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812142592 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 14 02:52:17.596596 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.596584    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 14 02:52:17.596767 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.596711    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 14 02:52:17.600427 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.600399    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 14 02:52:17.600589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.600429    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-244.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 14 02:52:17.601408 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.601396    2579 topology_manager.go:138] "Creating topology manager with none policy"
Sep 14 02:52:17.601474 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.601414    2579 container_manager_linux.go:309] "Creating device plugin manager"
Sep 14 02:52:17.601474 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.601430    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 02:52:17.602271 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.602258    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 02:52:17.603436 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.603424    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 14 02:52:17.607860 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.607846    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 02:52:17.607987 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.607976    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 14 02:52:17.608042 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.608012    2579 kubelet.go:486] "Attempting to sync node with API server"
Sep 14 02:52:17.608042 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.608035    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 14 02:52:17.608139 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.608058    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 14 02:52:17.608139 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.608071    2579 kubelet.go:398] "Adding apiserver pod source"
Sep 14 02:52:17.608139 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.608082    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 14 02:52:17.608465 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.608450    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 02:52:17.608523 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.608474    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 02:52:17.610923 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.610907    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 14 02:52:17.613908 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.613893    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 14 02:52:17.613974 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.613923    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 14 02:52:17.616066 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.616053    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 14 02:52:17.616104 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.616069    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 14 02:52:17.616104 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.616076    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 14 02:52:17.616104 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.616081    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 14 02:52:17.616104 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.616090    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 14 02:52:17.616104 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.616096    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 14 02:52:17.616255 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.616108    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 14 02:52:17.616255 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.616114    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 14 02:52:17.616255 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.616120    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 14 02:52:17.616255 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.616125    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 14 02:52:17.616255 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.616134    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 14 02:52:17.616255 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.616139    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 14 02:52:17.617035 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.617025    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 14 02:52:17.617064 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.617035    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 14 02:52:17.619839 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.619821    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-244.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 14 02:52:17.620516 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:17.620478    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 14 02:52:17.620516 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:17.620496    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-244.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 14 02:52:17.621001 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.620987    2579 server.go:1268] "Started kubelet"
Sep 14 02:52:17.624675 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.621064    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 14 02:52:17.624887 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.624867    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-96xrj"
Sep 14 02:52:17.625495 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.624693    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 14 02:52:17.625495 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.625112    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 14 02:52:17.625388 ip-10-0-138-244 systemd[1]: Started Kubernetes Kubelet.
Sep 14 02:52:17.627436 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.627419    2579 server.go:310] "Adding debug handlers to kubelet server"
Sep 14 02:52:17.628485 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:17.628465    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 14 02:52:17.628644 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.628626    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 14 02:52:17.630229 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:17.629156    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-244.ec2.internal.18d5104a272db13c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-244.ec2.internal,UID:ip-10-0-138-244.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-244.ec2.internal,},FirstTimestamp:2026-09-14 02:52:17.620570428 +0000 UTC m=+0.406632332,LastTimestamp:2026-09-14 02:52:17.620570428 +0000 UTC m=+0.406632332,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-244.ec2.internal,}"
Sep 14 02:52:17.630582 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.630560    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-96xrj"
Sep 14 02:52:17.636922 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.636905    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 14 02:52:17.637613 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.637588    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 14 02:52:17.637767 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:17.637752    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Sep 14 02:52:17.638315 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.638298    2579 factory.go:55] Registering systemd factory
Sep 14 02:52:17.638400 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.638319    2579 factory.go:223] Registration of the systemd container factory successfully
Sep 14 02:52:17.638400 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.638372    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 14 02:52:17.638400 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.638375    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 14 02:52:17.638400 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.638397    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 14 02:52:17.638537 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.638450    2579 reconstruct.go:97] "Volume reconstruction finished"
Sep 14 02:52:17.638537 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.638460    2579 reconciler.go:29] "Reconciler: start to sync state"
Sep 14 02:52:17.638596 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.638587    2579 factory.go:153] Registering CRI-O factory
Sep 14 02:52:17.638626 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.638599    2579 factory.go:223] Registration of the crio container factory successfully
Sep 14 02:52:17.638661 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.638650    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 14 02:52:17.638695 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.638671    2579 factory.go:103] Registering Raw factory
Sep 14 02:52:17.638695 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.638684    2579 manager.go:1196] Started watching for new ooms in manager
Sep 14 02:52:17.639080 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.639068    2579 manager.go:349] Starting recovery of all containers
Sep 14 02:52:17.646893 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.646878    2579 manager.go:354] Recovery completed
Sep 14 02:52:17.650976 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.650955    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 02:52:17.651392 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.651379    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:52:17.653554 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.653541    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:52:17.653607 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.653567    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:52:17.653607 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.653579    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:52:17.654043 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:17.654028    2579 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-244.ec2.internal\" not found" node="ip-10-0-138-244.ec2.internal"
Sep 14 02:52:17.654142 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.654129    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 14 02:52:17.654194 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.654145    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:52:17.654194 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.654143    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 14 02:52:17.654257 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.654168    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 02:52:17.654891 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.654878    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:52:17.654947 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.654900    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:52:17.654947 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.654909    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:52:17.656337 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.656325    2579 policy_none.go:49] "None policy: Start"
Sep 14 02:52:17.656381 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.656341    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 14 02:52:17.656381 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.656351    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 14 02:52:17.658386 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.658372    2579 policy_none.go:47] "Start"
Sep 14 02:52:17.693884 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.693870    2579 manager.go:337] "Starting Device Plugin manager"
Sep 14 02:52:17.718363 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:17.693903    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 14 02:52:17.718363 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.693913    2579 server.go:87] "Starting device plugin registration server"
Sep 14 02:52:17.718363 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.694087    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 14 02:52:17.718363 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.694096    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 14 02:52:17.718363 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.694202    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 14 02:52:17.718363 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.694283    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 14 02:52:17.718363 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.694291    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 14 02:52:17.718363 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:17.694749    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 14 02:52:17.718363 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:17.694789    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-244.ec2.internal\" not found"
Sep 14 02:52:17.735639 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.735620    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 14 02:52:17.736890 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.736872    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 14 02:52:17.736963 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.736912    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 14 02:52:17.736963 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.736931    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 14 02:52:17.737055 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:17.736968    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 14 02:52:17.739632 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.739612    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 02:52:17.794742 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.794662    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:52:17.795573 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.795558    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:52:17.795642 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.795588    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:52:17.795642 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.795598    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:52:17.795642 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.795624    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-244.ec2.internal"
Sep 14 02:52:17.802285 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.802271    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-244.ec2.internal"
Sep 14 02:52:17.802325 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:17.802293    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-244.ec2.internal\": node \"ip-10-0-138-244.ec2.internal\" not found"
Sep 14 02:52:17.820433 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:17.820411    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Sep 14 02:52:17.837865 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.837843    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal"]
Sep 14 02:52:17.837935 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.837899    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:52:17.839488 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.839472    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ffe75fa59184a13c6e136c98cae952e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal\" (UID: \"ffe75fa59184a13c6e136c98cae952e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Sep 14 02:52:17.839580 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.839494    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffe75fa59184a13c6e136c98cae952e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal\" (UID: \"ffe75fa59184a13c6e136c98cae952e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Sep 14 02:52:17.839700 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.839688    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:52:17.839742 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.839713    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:52:17.839742 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.839728    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:52:17.841833 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.841822    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:52:17.841970 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.841957    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Sep 14 02:52:17.842009 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.841987    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:52:17.842901 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.842887    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:52:17.842948 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.842919    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:52:17.842948 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.842929    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:52:17.843020 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.842891    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:52:17.843020 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.842998    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:52:17.843020 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.843010    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:52:17.845086 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.845070    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal"
Sep 14 02:52:17.845126 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.845108    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 02:52:17.845795 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.845783    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 02:52:17.845855 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.845806    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 02:52:17.845855 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.845815    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeHasSufficientPID"
Sep 14 02:52:17.867734 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:17.867713    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-244.ec2.internal\" not found" node="ip-10-0-138-244.ec2.internal"
Sep 14 02:52:17.871126 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:17.871110    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-244.ec2.internal\" not found" node="ip-10-0-138-244.ec2.internal"
Sep 14 02:52:17.920888 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:17.920866    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Sep 14 02:52:17.940264 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.940242    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ffe75fa59184a13c6e136c98cae952e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal\" (UID: \"ffe75fa59184a13c6e136c98cae952e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Sep 14 02:52:17.940333 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.940267    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffe75fa59184a13c6e136c98cae952e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal\" (UID: \"ffe75fa59184a13c6e136c98cae952e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Sep 14 02:52:17.940333 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.940286    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be05b9bf67903a6091aac71677a6fb10-config\") pod \"kube-apiserver-proxy-ip-10-0-138-244.ec2.internal\" (UID: \"be05b9bf67903a6091aac71677a6fb10\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal"
Sep 14 02:52:17.940444 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.940352    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffe75fa59184a13c6e136c98cae952e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal\" (UID: \"ffe75fa59184a13c6e136c98cae952e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Sep 14 02:52:17.940444 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:17.940356    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ffe75fa59184a13c6e136c98cae952e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal\" (UID: \"ffe75fa59184a13c6e136c98cae952e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Sep 14 02:52:18.021978 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:18.021942    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Sep 14 02:52:18.040426 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:18.040404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be05b9bf67903a6091aac71677a6fb10-config\") pod \"kube-apiserver-proxy-ip-10-0-138-244.ec2.internal\" (UID: \"be05b9bf67903a6091aac71677a6fb10\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal"
Sep 14 02:52:18.040496 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:18.040464    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be05b9bf67903a6091aac71677a6fb10-config\") pod \"kube-apiserver-proxy-ip-10-0-138-244.ec2.internal\" (UID: \"be05b9bf67903a6091aac71677a6fb10\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal"
Sep 14 02:52:18.122859 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:18.122796    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Sep 14 02:52:18.171289 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:18.171273    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Sep 14 02:52:18.173445 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:18.173427    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal"
Sep 14 02:52:18.223890 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:18.223864    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Sep 14 02:52:18.324362 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:18.324333    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Sep 14 02:52:18.424853 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:18.424792    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Sep 14 02:52:18.525301 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:18.525273    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Sep 14 02:52:18.554656 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:18.554640    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 14 02:52:18.554946 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:18.554763    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 02:52:18.554946 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:18.554786    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 02:52:18.626342 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:18.626199    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Sep 14 02:52:18.638353 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:18.638296    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-13 02:47:18 +0000 UTC" deadline="2028-05-21 00:00:08.768175519 +0000 UTC"
Sep 14 02:52:18.638353 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:18.638352    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14757h7m50.129828228s"
Sep 14 02:52:18.704357 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:18.704332    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffe75fa59184a13c6e136c98cae952e6.slice/crio-063870950f28567986332d6e67ed2a23a3cc7478bc807be4be3d147b9d0fe6c3 WatchSource:0}: Error finding container 063870950f28567986332d6e67ed2a23a3cc7478bc807be4be3d147b9d0fe6c3: Status 404 returned error can't find the container with id 063870950f28567986332d6e67ed2a23a3cc7478bc807be4be3d147b9d0fe6c3
Sep 14 02:52:18.705273 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:18.705243    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe05b9bf67903a6091aac71677a6fb10.slice/crio-29017169c83dae4c2dd3a0be66bacabb22a45fcbbae6c7d81437b251d791f7e8 WatchSource:0}: Error finding container 29017169c83dae4c2dd3a0be66bacabb22a45fcbbae6c7d81437b251d791f7e8: Status 404 returned error can't find the container with id 29017169c83dae4c2dd3a0be66bacabb22a45fcbbae6c7d81437b251d791f7e8
Sep 14 02:52:18.709602 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:18.709579    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 02:52:18.726784 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:18.726763    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Sep 14 02:52:18.739773 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:18.739733    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal" event={"ID":"ffe75fa59184a13c6e136c98cae952e6","Type":"ContainerStarted","Data":"063870950f28567986332d6e67ed2a23a3cc7478bc807be4be3d147b9d0fe6c3"}
Sep 14 02:52:18.740674 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:18.740652    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal" event={"ID":"be05b9bf67903a6091aac71677a6fb10","Type":"ContainerStarted","Data":"29017169c83dae4c2dd3a0be66bacabb22a45fcbbae6c7d81437b251d791f7e8"}
Sep 14 02:52:18.827009 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:18.826985    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Sep 14 02:52:18.927573 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:18.927530    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Sep 14 02:52:19.028637 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:19.028566    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-244.ec2.internal\" not found"
Sep 14 02:52:19.079497 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.079465    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 02:52:19.138463 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.138221    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal"
Sep 14 02:52:19.155187 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.155147    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 02:52:19.156839 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.156814    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal"
Sep 14 02:52:19.166061 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.166033    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 02:52:19.191236 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.191079    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 02:52:19.609952 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.609925    2579 apiserver.go:52] "Watching apiserver"
Sep 14 02:52:19.622338 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.622312    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 14 02:52:19.623169 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.623136    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-r2jvl","openshift-dns/node-resolver-mhwbd","openshift-multus/multus-q9w7g","openshift-network-operator/iptables-alerter-f6w6v","openshift-ovn-kubernetes/ovnkube-node-zvqjb","kube-system/global-pull-secret-syncer-h5wrd","kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc","openshift-image-registry/node-ca-gp9kv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal","openshift-multus/multus-additional-cni-plugins-4bxmf","openshift-multus/network-metrics-daemon-ct7gq","openshift-network-diagnostics/network-check-target-484dv","kube-system/konnectivity-agent-6wg97"]
Sep 14 02:52:19.626079 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.626055    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:19.626245 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:19.626180    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:19.628409 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.628384    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mhwbd"
Sep 14 02:52:19.630976 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.630948    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.631531 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.631510    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 14 02:52:19.631674 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.631606    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-n8p5b\""
Sep 14 02:52:19.631674 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.631510    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 14 02:52:19.633436 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.633417    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f6w6v"
Sep 14 02:52:19.634735 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.634711    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 14 02:52:19.634955 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.634934    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 14 02:52:19.635047 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.635029    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 14 02:52:19.635110 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.635058    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 14 02:52:19.636100 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.636070    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gchdp\""
Sep 14 02:52:19.636221 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.636202    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 14 02:52:19.636438 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.636417    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 14 02:52:19.636580 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.636565    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 14 02:52:19.636681 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.636663    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qll24\""
Sep 14 02:52:19.637081 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.636917    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.638024 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.638005    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 14 02:52:19.638995 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.638976    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:19.639097 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:19.639052    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:19.643108 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.643085    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 14 02:52:19.643208 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.643140    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 14 02:52:19.643282 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.643091    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 14 02:52:19.643694 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.643671    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.644112 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.644097    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 14 02:52:19.644303 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.644285    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 14 02:52:19.644411 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.644390    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qxr9b\""
Sep 14 02:52:19.644761 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.644566    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 14 02:52:19.646072 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.646036    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.649642 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.649610    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 14 02:52:19.650035 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.649999    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 14 02:52:19.650324 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.650304    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-n9hw9\""
Sep 14 02:52:19.650601 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.650584    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8sw8k\""
Sep 14 02:52:19.650852 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.650836    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 14 02:52:19.651019 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651003    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 14 02:52:19.651226 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651208    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 14 02:52:19.651529 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651488    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-slash\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.651529 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651523    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-run-ovn\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.651675 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651550    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret\") pod \"global-pull-secret-syncer-h5wrd\" (UID: \"2407936d-8e27-4f88-93a8-9dca1a2b80b9\") " pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:19.651675 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651579    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4wtg\" (UniqueName: \"kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg\") pod \"network-check-target-484dv\" (UID: \"fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4\") " pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:19.651675 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651633    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-etc-kubernetes\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.651838 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651677    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3311ebca-8cba-4235-b0bd-a9908b50c143-host-slash\") pod \"iptables-alerter-f6w6v\" (UID: \"3311ebca-8cba-4235-b0bd-a9908b50c143\") " pod="openshift-network-operator/iptables-alerter-f6w6v"
Sep 14 02:52:19.651838 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651724    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-run-netns\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.651838 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651776    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-system-cni-dir\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.651838 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651832    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-run-netns\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.652029 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651866    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-run-openvswitch\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.652029 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651892    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-cni-bin\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.652029 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651916    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c6897cf2-5e8d-49f9-a31d-a408a10acbed-env-overrides\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.652029 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651932    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-cnibin\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.652029 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651956    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/45b773f0-c1aa-456b-b7e0-9427682f4bd9-cni-binary-copy\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.652029 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651937    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gp9kv"
Sep 14 02:52:19.652029 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.651980    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-var-lib-kubelet\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.652029 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652004    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drmrk\" (UniqueName: \"kubernetes.io/projected/3311ebca-8cba-4235-b0bd-a9908b50c143-kube-api-access-drmrk\") pod \"iptables-alerter-f6w6v\" (UID: \"3311ebca-8cba-4235-b0bd-a9908b50c143\") " pod="openshift-network-operator/iptables-alerter-f6w6v"
Sep 14 02:52:19.652433 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652051    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-var-lib-openvswitch\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.652433 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652091    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-log-socket\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.652433 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652122    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2407936d-8e27-4f88-93a8-9dca1a2b80b9-kubelet-config\") pod \"global-pull-secret-syncer-h5wrd\" (UID: \"2407936d-8e27-4f88-93a8-9dca1a2b80b9\") " pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:19.652433 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652148    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-run-multus-certs\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.652433 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652190    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-kubelet\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.652433 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652237    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-cni-netd\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.652433 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652257    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c6897cf2-5e8d-49f9-a31d-a408a10acbed-ovnkube-script-lib\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.652433 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652295    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbwxt\" (UniqueName: \"kubernetes.io/projected/c6897cf2-5e8d-49f9-a31d-a408a10acbed-kube-api-access-jbwxt\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.652433 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652327    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlwkr\" (UniqueName: \"kubernetes.io/projected/1a65bc4c-0400-4b9f-ab60-75b0ff402476-kube-api-access-vlwkr\") pod \"node-resolver-mhwbd\" (UID: \"1a65bc4c-0400-4b9f-ab60-75b0ff402476\") " pod="openshift-dns/node-resolver-mhwbd"
Sep 14 02:52:19.652433 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652349    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-run-k8s-cni-cncf-io\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.652433 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652373    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-multus-conf-dir\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.652433 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652388    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/45b773f0-c1aa-456b-b7e0-9427682f4bd9-multus-daemon-config\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.652433 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652404    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-etc-openvswitch\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.652979 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652470    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-node-log\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.652979 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652493    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.652979 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652524    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c6897cf2-5e8d-49f9-a31d-a408a10acbed-ovnkube-config\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.652979 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652568    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a65bc4c-0400-4b9f-ab60-75b0ff402476-tmp-dir\") pod \"node-resolver-mhwbd\" (UID: \"1a65bc4c-0400-4b9f-ab60-75b0ff402476\") " pod="openshift-dns/node-resolver-mhwbd"
Sep 14 02:52:19.652979 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652612    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-multus-socket-dir-parent\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.652979 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652646    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hppn8\" (UniqueName: \"kubernetes.io/projected/45b773f0-c1aa-456b-b7e0-9427682f4bd9-kube-api-access-hppn8\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.652979 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652676    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-systemd-units\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.652979 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652701    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-run-systemd\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.652979 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652723    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-run-ovn-kubernetes\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.652979 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652740    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2407936d-8e27-4f88-93a8-9dca1a2b80b9-dbus\") pod \"global-pull-secret-syncer-h5wrd\" (UID: \"2407936d-8e27-4f88-93a8-9dca1a2b80b9\") " pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:19.652979 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652756    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-multus-cni-dir\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.652979 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652800    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-var-lib-cni-multus\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.652979 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652863    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3311ebca-8cba-4235-b0bd-a9908b50c143-iptables-alerter-script\") pod \"iptables-alerter-f6w6v\" (UID: \"3311ebca-8cba-4235-b0bd-a9908b50c143\") " pod="openshift-network-operator/iptables-alerter-f6w6v"
Sep 14 02:52:19.653672 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.652893    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c6897cf2-5e8d-49f9-a31d-a408a10acbed-ovn-node-metrics-cert\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.653672 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.653479    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1a65bc4c-0400-4b9f-ab60-75b0ff402476-hosts-file\") pod \"node-resolver-mhwbd\" (UID: \"1a65bc4c-0400-4b9f-ab60-75b0ff402476\") " pod="openshift-dns/node-resolver-mhwbd"
Sep 14 02:52:19.653672 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.653508    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-var-lib-cni-bin\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.653672 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.653581    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-hostroot\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.655840 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.655678    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fbmbl\""
Sep 14 02:52:19.655840 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.655710    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 14 02:52:19.655840 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.655724    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 14 02:52:19.656016 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.655836    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 14 02:52:19.658297 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.658279    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6wg97"
Sep 14 02:52:19.660805 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.660779    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.661701 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.661678    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 02:52:19.663340 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.663318    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:19.663420 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:19.663406    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:19.664573 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.664549    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qrkf7\""
Sep 14 02:52:19.665464 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.665355    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 14 02:52:19.665591 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.665504    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 14 02:52:19.665691 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.665662    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8kxlt\""
Sep 14 02:52:19.668685 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.668663    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 14 02:52:19.668878 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.668711    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 14 02:52:19.696606 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.696586    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hc84x"
Sep 14 02:52:19.704231 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.704199    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hc84x"
Sep 14 02:52:19.739523 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.739500    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 14 02:52:19.754316 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754288    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/278ffea9-7be2-48a9-b480-451029260897-tmp\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.754439 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754342    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clnq6\" (UniqueName: \"kubernetes.io/projected/cae07d88-4fe4-41c8-b2f9-2b8abb3bc353-kube-api-access-clnq6\") pod \"node-ca-gp9kv\" (UID: \"cae07d88-4fe4-41c8-b2f9-2b8abb3bc353\") " pod="openshift-image-registry/node-ca-gp9kv"
Sep 14 02:52:19.754439 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754373    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-run-openvswitch\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.754439 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754399    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-cnibin\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.754439 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754421    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/45b773f0-c1aa-456b-b7e0-9427682f4bd9-cni-binary-copy\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.754636 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754445    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-var-lib-kubelet\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.754636 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754467    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-run-openvswitch\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.754636 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754471    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drmrk\" (UniqueName: \"kubernetes.io/projected/3311ebca-8cba-4235-b0bd-a9908b50c143-kube-api-access-drmrk\") pod \"iptables-alerter-f6w6v\" (UID: \"3311ebca-8cba-4235-b0bd-a9908b50c143\") " pod="openshift-network-operator/iptables-alerter-f6w6v"
Sep 14 02:52:19.754636 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754524    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-sys-fs\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.754636 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754548    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cae07d88-4fe4-41c8-b2f9-2b8abb3bc353-host\") pod \"node-ca-gp9kv\" (UID: \"cae07d88-4fe4-41c8-b2f9-2b8abb3bc353\") " pod="openshift-image-registry/node-ca-gp9kv"
Sep 14 02:52:19.754636 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754558    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-var-lib-kubelet\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.754636 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754543    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-cnibin\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.754636 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754579    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnqhb\" (UniqueName: \"kubernetes.io/projected/35e90f11-d246-46e8-aa8b-ca9448f164e3-kube-api-access-jnqhb\") pod \"network-metrics-daemon-ct7gq\" (UID: \"35e90f11-d246-46e8-aa8b-ca9448f164e3\") " pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:19.754636 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754632    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-log-socket\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754672    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-run-multus-certs\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754731    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-run-multus-certs\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754741    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-log-socket\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754769    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-sysconfig\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754806    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-kubelet\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754826    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c6897cf2-5e8d-49f9-a31d-a408a10acbed-ovnkube-script-lib\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754846    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlwkr\" (UniqueName: \"kubernetes.io/projected/1a65bc4c-0400-4b9f-ab60-75b0ff402476-kube-api-access-vlwkr\") pod \"node-resolver-mhwbd\" (UID: \"1a65bc4c-0400-4b9f-ab60-75b0ff402476\") " pod="openshift-dns/node-resolver-mhwbd"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754853    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-kubelet\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754861    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-multus-conf-dir\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754877    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/45b773f0-c1aa-456b-b7e0-9427682f4bd9-multus-daemon-config\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754896    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-systemd\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754916    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f0a6b9f0-9453-482b-aa52-a8526bdfcf15-konnectivity-ca\") pod \"konnectivity-agent-6wg97\" (UID: \"f0a6b9f0-9453-482b-aa52-a8526bdfcf15\") " pod="kube-system/konnectivity-agent-6wg97"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754951    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-multus-conf-dir\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754961    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-etc-openvswitch\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.754985    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-node-log\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755077    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-etc-openvswitch\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755097    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-node-log\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.755172 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755107    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/45b773f0-c1aa-456b-b7e0-9427682f4bd9-cni-binary-copy\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755116    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755147    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c6897cf2-5e8d-49f9-a31d-a408a10acbed-ovnkube-config\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755188    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755173    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-multus-socket-dir-parent\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755235    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-systemd-units\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755254    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-run-ovn-kubernetes\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755287    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2407936d-8e27-4f88-93a8-9dca1a2b80b9-dbus\") pod \"global-pull-secret-syncer-h5wrd\" (UID: \"2407936d-8e27-4f88-93a8-9dca1a2b80b9\") " pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755296    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-systemd-units\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755303    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-multus-cni-dir\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755352    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-multus-socket-dir-parent\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755379    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-var-lib-cni-multus\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755407    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9t2k\" (UniqueName: \"kubernetes.io/projected/e343d514-241a-41a4-86a6-bbdb05394f79-kube-api-access-g9t2k\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755445    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2407936d-8e27-4f88-93a8-9dca1a2b80b9-dbus\") pod \"global-pull-secret-syncer-h5wrd\" (UID: \"2407936d-8e27-4f88-93a8-9dca1a2b80b9\") " pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755445    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-multus-cni-dir\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755453    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs\") pod \"network-metrics-daemon-ct7gq\" (UID: \"35e90f11-d246-46e8-aa8b-ca9448f164e3\") " pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755472    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/45b773f0-c1aa-456b-b7e0-9427682f4bd9-multus-daemon-config\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.756017 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755475    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-var-lib-cni-multus\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755497    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-run-ovn-kubernetes\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755505    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c6897cf2-5e8d-49f9-a31d-a408a10acbed-ovn-node-metrics-cert\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755529    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret\") pod \"global-pull-secret-syncer-h5wrd\" (UID: \"2407936d-8e27-4f88-93a8-9dca1a2b80b9\") " pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755455    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c6897cf2-5e8d-49f9-a31d-a408a10acbed-ovnkube-script-lib\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755555    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-sysctl-conf\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755572    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-device-dir\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755612    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c6897cf2-5e8d-49f9-a31d-a408a10acbed-ovnkube-config\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:19.755629    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:19.755693    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret podName:2407936d-8e27-4f88-93a8-9dca1a2b80b9 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:20.255672659 +0000 UTC m=+3.041734571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret") pod "global-pull-secret-syncer-h5wrd" (UID: "2407936d-8e27-4f88-93a8-9dca1a2b80b9") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755632    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-system-cni-dir\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755713    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-system-cni-dir\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-cni-bin\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755756    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c6897cf2-5e8d-49f9-a31d-a408a10acbed-env-overrides\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755784    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-modprobe-d\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755794    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-cni-bin\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755809    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-sys\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.756787 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755857    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/278ffea9-7be2-48a9-b480-451029260897-etc-tuned\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755885    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-etc-selinux\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755847    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755915    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6f411fd9-d361-42cf-8268-d17b6c94dff5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755944    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-var-lib-openvswitch\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755973    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-var-lib-openvswitch\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.755992    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2407936d-8e27-4f88-93a8-9dca1a2b80b9-kubelet-config\") pod \"global-pull-secret-syncer-h5wrd\" (UID: \"2407936d-8e27-4f88-93a8-9dca1a2b80b9\") " pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756026    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756060    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2407936d-8e27-4f88-93a8-9dca1a2b80b9-kubelet-config\") pod \"global-pull-secret-syncer-h5wrd\" (UID: \"2407936d-8e27-4f88-93a8-9dca1a2b80b9\") " pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756105    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-cni-netd\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756125    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-cni-netd\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756132    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbwxt\" (UniqueName: \"kubernetes.io/projected/c6897cf2-5e8d-49f9-a31d-a408a10acbed-kube-api-access-jbwxt\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756158    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-run-k8s-cni-cncf-io\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756160    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c6897cf2-5e8d-49f9-a31d-a408a10acbed-env-overrides\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756187    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-sysctl-d\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756196    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-run-k8s-cni-cncf-io\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756224    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-run\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.757424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756246    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-var-lib-kubelet\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756262    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljw9p\" (UniqueName: \"kubernetes.io/projected/278ffea9-7be2-48a9-b480-451029260897-kube-api-access-ljw9p\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756278    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f411fd9-d361-42cf-8268-d17b6c94dff5-system-cni-dir\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a65bc4c-0400-4b9f-ab60-75b0ff402476-tmp-dir\") pod \"node-resolver-mhwbd\" (UID: \"1a65bc4c-0400-4b9f-ab60-75b0ff402476\") " pod="openshift-dns/node-resolver-mhwbd"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756354    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hppn8\" (UniqueName: \"kubernetes.io/projected/45b773f0-c1aa-456b-b7e0-9427682f4bd9-kube-api-access-hppn8\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756373    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-socket-dir\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756389    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f411fd9-d361-42cf-8268-d17b6c94dff5-cnibin\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756411    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f411fd9-d361-42cf-8268-d17b6c94dff5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756429    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vfzs\" (UniqueName: \"kubernetes.io/projected/6f411fd9-d361-42cf-8268-d17b6c94dff5-kube-api-access-6vfzs\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756445    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-run-systemd\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756463    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3311ebca-8cba-4235-b0bd-a9908b50c143-iptables-alerter-script\") pod \"iptables-alerter-f6w6v\" (UID: \"3311ebca-8cba-4235-b0bd-a9908b50c143\") " pod="openshift-network-operator/iptables-alerter-f6w6v"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756487    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-kubernetes\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756512    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-lib-modules\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756536    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-registration-dir\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756561    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f0a6b9f0-9453-482b-aa52-a8526bdfcf15-agent-certs\") pod \"konnectivity-agent-6wg97\" (UID: \"f0a6b9f0-9453-482b-aa52-a8526bdfcf15\") " pod="kube-system/konnectivity-agent-6wg97"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756539    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-run-systemd\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756588    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1a65bc4c-0400-4b9f-ab60-75b0ff402476-hosts-file\") pod \"node-resolver-mhwbd\" (UID: \"1a65bc4c-0400-4b9f-ab60-75b0ff402476\") " pod="openshift-dns/node-resolver-mhwbd"
Sep 14 02:52:19.757967 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756604    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a65bc4c-0400-4b9f-ab60-75b0ff402476-tmp-dir\") pod \"node-resolver-mhwbd\" (UID: \"1a65bc4c-0400-4b9f-ab60-75b0ff402476\") " pod="openshift-dns/node-resolver-mhwbd"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756625    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-var-lib-cni-bin\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756651    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1a65bc4c-0400-4b9f-ab60-75b0ff402476-hosts-file\") pod \"node-resolver-mhwbd\" (UID: \"1a65bc4c-0400-4b9f-ab60-75b0ff402476\") " pod="openshift-dns/node-resolver-mhwbd"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756651    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-hostroot\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756682    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-var-lib-cni-bin\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756695    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-hostroot\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756695    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cae07d88-4fe4-41c8-b2f9-2b8abb3bc353-serviceca\") pod \"node-ca-gp9kv\" (UID: \"cae07d88-4fe4-41c8-b2f9-2b8abb3bc353\") " pod="openshift-image-registry/node-ca-gp9kv"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756737    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f411fd9-d361-42cf-8268-d17b6c94dff5-cni-binary-copy\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756765    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6f411fd9-d361-42cf-8268-d17b6c94dff5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756797    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-slash\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756822    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-run-ovn\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756851    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4wtg\" (UniqueName: \"kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg\") pod \"network-check-target-484dv\" (UID: \"fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4\") " pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756861    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-slash\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756880    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-etc-kubernetes\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756881    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-run-ovn\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756913    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3311ebca-8cba-4235-b0bd-a9908b50c143-host-slash\") pod \"iptables-alerter-f6w6v\" (UID: \"3311ebca-8cba-4235-b0bd-a9908b50c143\") " pod="openshift-network-operator/iptables-alerter-f6w6v"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756960    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-run-netns\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756964    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-etc-kubernetes\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.758521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.757000    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-run-netns\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.759124 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.757014    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3311ebca-8cba-4235-b0bd-a9908b50c143-host-slash\") pod \"iptables-alerter-f6w6v\" (UID: \"3311ebca-8cba-4235-b0bd-a9908b50c143\") " pod="openshift-network-operator/iptables-alerter-f6w6v"
Sep 14 02:52:19.759124 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.756996    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3311ebca-8cba-4235-b0bd-a9908b50c143-iptables-alerter-script\") pod \"iptables-alerter-f6w6v\" (UID: \"3311ebca-8cba-4235-b0bd-a9908b50c143\") " pod="openshift-network-operator/iptables-alerter-f6w6v"
Sep 14 02:52:19.759124 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.757017    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c6897cf2-5e8d-49f9-a31d-a408a10acbed-host-run-netns\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.759124 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.757033    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-host\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.759124 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.757100    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45b773f0-c1aa-456b-b7e0-9427682f4bd9-host-run-netns\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.760119 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.760099    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c6897cf2-5e8d-49f9-a31d-a408a10acbed-ovn-node-metrics-cert\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.766750 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:19.766722    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:52:19.766857 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:19.766752    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:52:19.766857 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:19.766767    2579 projected.go:196] Error preparing data for projected volume kube-api-access-m4wtg for pod openshift-network-diagnostics/network-check-target-484dv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:52:19.766980 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:19.766870    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg podName:fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:20.266851391 +0000 UTC m=+3.052913303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-m4wtg" (UniqueName: "kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg") pod "network-check-target-484dv" (UID: "fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:52:19.767479 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.767442    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlwkr\" (UniqueName: \"kubernetes.io/projected/1a65bc4c-0400-4b9f-ab60-75b0ff402476-kube-api-access-vlwkr\") pod \"node-resolver-mhwbd\" (UID: \"1a65bc4c-0400-4b9f-ab60-75b0ff402476\") " pod="openshift-dns/node-resolver-mhwbd"
Sep 14 02:52:19.767612 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.767480    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drmrk\" (UniqueName: \"kubernetes.io/projected/3311ebca-8cba-4235-b0bd-a9908b50c143-kube-api-access-drmrk\") pod \"iptables-alerter-f6w6v\" (UID: \"3311ebca-8cba-4235-b0bd-a9908b50c143\") " pod="openshift-network-operator/iptables-alerter-f6w6v"
Sep 14 02:52:19.769717 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.769695    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hppn8\" (UniqueName: \"kubernetes.io/projected/45b773f0-c1aa-456b-b7e0-9427682f4bd9-kube-api-access-hppn8\") pod \"multus-q9w7g\" (UID: \"45b773f0-c1aa-456b-b7e0-9427682f4bd9\") " pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.770069 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.770047    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbwxt\" (UniqueName: \"kubernetes.io/projected/c6897cf2-5e8d-49f9-a31d-a408a10acbed-kube-api-access-jbwxt\") pod \"ovnkube-node-zvqjb\" (UID: \"c6897cf2-5e8d-49f9-a31d-a408a10acbed\") " pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.857954 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.857918    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-modprobe-d\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.858133 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.857962    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-sys\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.858133 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.857985    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/278ffea9-7be2-48a9-b480-451029260897-etc-tuned\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.858133 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858010    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-etc-selinux\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.858133 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858036    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6f411fd9-d361-42cf-8268-d17b6c94dff5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.858133 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858061    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.858133 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858073    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-sys\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.858133 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858087    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-sysctl-d\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.858133 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858118    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-etc-selinux\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858140    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858111    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-modprobe-d\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858174    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-run\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858200    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-var-lib-kubelet\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858220    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-sysctl-d\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858252    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-run\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858273    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-var-lib-kubelet\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858291    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljw9p\" (UniqueName: \"kubernetes.io/projected/278ffea9-7be2-48a9-b480-451029260897-kube-api-access-ljw9p\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858320    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f411fd9-d361-42cf-8268-d17b6c94dff5-system-cni-dir\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858338    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-socket-dir\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f411fd9-d361-42cf-8268-d17b6c94dff5-cnibin\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858383    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f411fd9-d361-42cf-8268-d17b6c94dff5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858397    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f411fd9-d361-42cf-8268-d17b6c94dff5-system-cni-dir\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vfzs\" (UniqueName: \"kubernetes.io/projected/6f411fd9-d361-42cf-8268-d17b6c94dff5-kube-api-access-6vfzs\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858438    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-kubernetes\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858440    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f411fd9-d361-42cf-8268-d17b6c94dff5-cnibin\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.858605 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858471    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-lib-modules\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858492    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-kubernetes\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858497    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-registration-dir\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858521    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f0a6b9f0-9453-482b-aa52-a8526bdfcf15-agent-certs\") pod \"konnectivity-agent-6wg97\" (UID: \"f0a6b9f0-9453-482b-aa52-a8526bdfcf15\") " pod="kube-system/konnectivity-agent-6wg97"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858549    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cae07d88-4fe4-41c8-b2f9-2b8abb3bc353-serviceca\") pod \"node-ca-gp9kv\" (UID: \"cae07d88-4fe4-41c8-b2f9-2b8abb3bc353\") " pod="openshift-image-registry/node-ca-gp9kv"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858571    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f411fd9-d361-42cf-8268-d17b6c94dff5-cni-binary-copy\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858596    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6f411fd9-d361-42cf-8268-d17b6c94dff5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858626    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-lib-modules\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858640    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f411fd9-d361-42cf-8268-d17b6c94dff5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858691    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-registration-dir\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858727    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-socket-dir\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858703    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6f411fd9-d361-42cf-8268-d17b6c94dff5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858642    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-host\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858956    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/278ffea9-7be2-48a9-b480-451029260897-tmp\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.858985    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clnq6\" (UniqueName: \"kubernetes.io/projected/cae07d88-4fe4-41c8-b2f9-2b8abb3bc353-kube-api-access-clnq6\") pod \"node-ca-gp9kv\" (UID: \"cae07d88-4fe4-41c8-b2f9-2b8abb3bc353\") " pod="openshift-image-registry/node-ca-gp9kv"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859015    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-sys-fs\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859058    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cae07d88-4fe4-41c8-b2f9-2b8abb3bc353-host\") pod \"node-ca-gp9kv\" (UID: \"cae07d88-4fe4-41c8-b2f9-2b8abb3bc353\") " pod="openshift-image-registry/node-ca-gp9kv"
Sep 14 02:52:19.859589 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859083    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnqhb\" (UniqueName: \"kubernetes.io/projected/35e90f11-d246-46e8-aa8b-ca9448f164e3-kube-api-access-jnqhb\") pod \"network-metrics-daemon-ct7gq\" (UID: \"35e90f11-d246-46e8-aa8b-ca9448f164e3\") " pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859112    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-sysconfig\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859141    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-systemd\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cae07d88-4fe4-41c8-b2f9-2b8abb3bc353-serviceca\") pod \"node-ca-gp9kv\" (UID: \"cae07d88-4fe4-41c8-b2f9-2b8abb3bc353\") " pod="openshift-image-registry/node-ca-gp9kv"
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f0a6b9f0-9453-482b-aa52-a8526bdfcf15-konnectivity-ca\") pod \"konnectivity-agent-6wg97\" (UID: \"f0a6b9f0-9453-482b-aa52-a8526bdfcf15\") " pod="kube-system/konnectivity-agent-6wg97"
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859219    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-sys-fs\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859221    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9t2k\" (UniqueName: \"kubernetes.io/projected/e343d514-241a-41a4-86a6-bbdb05394f79-kube-api-access-g9t2k\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859277    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs\") pod \"network-metrics-daemon-ct7gq\" (UID: \"35e90f11-d246-46e8-aa8b-ca9448f164e3\") " pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859317    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-sysctl-conf\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859340    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-device-dir\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859381    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f411fd9-d361-42cf-8268-d17b6c94dff5-cni-binary-copy\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859413    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e343d514-241a-41a4-86a6-bbdb05394f79-device-dir\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859431    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6f411fd9-d361-42cf-8268-d17b6c94dff5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859443    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-sysconfig\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859461    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-host\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859483    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-systemd\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:19.859530    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:52:19.860590 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:19.859598    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs podName:35e90f11-d246-46e8-aa8b-ca9448f164e3 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:20.359582168 +0000 UTC m=+3.145644064 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs") pod "network-metrics-daemon-ct7gq" (UID: "35e90f11-d246-46e8-aa8b-ca9448f164e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:52:19.861457 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859485    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cae07d88-4fe4-41c8-b2f9-2b8abb3bc353-host\") pod \"node-ca-gp9kv\" (UID: \"cae07d88-4fe4-41c8-b2f9-2b8abb3bc353\") " pod="openshift-image-registry/node-ca-gp9kv"
Sep 14 02:52:19.861457 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859712    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/278ffea9-7be2-48a9-b480-451029260897-etc-sysctl-conf\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.861457 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.859865    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f0a6b9f0-9453-482b-aa52-a8526bdfcf15-konnectivity-ca\") pod \"konnectivity-agent-6wg97\" (UID: \"f0a6b9f0-9453-482b-aa52-a8526bdfcf15\") " pod="kube-system/konnectivity-agent-6wg97"
Sep 14 02:52:19.861457 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.860794    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/278ffea9-7be2-48a9-b480-451029260897-etc-tuned\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.861816 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.861796    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f0a6b9f0-9453-482b-aa52-a8526bdfcf15-agent-certs\") pod \"konnectivity-agent-6wg97\" (UID: \"f0a6b9f0-9453-482b-aa52-a8526bdfcf15\") " pod="kube-system/konnectivity-agent-6wg97"
Sep 14 02:52:19.862146 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.862119    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/278ffea9-7be2-48a9-b480-451029260897-tmp\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.868522 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.868496    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljw9p\" (UniqueName: \"kubernetes.io/projected/278ffea9-7be2-48a9-b480-451029260897-kube-api-access-ljw9p\") pod \"tuned-r2jvl\" (UID: \"278ffea9-7be2-48a9-b480-451029260897\") " pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.871499 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.871475    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vfzs\" (UniqueName: \"kubernetes.io/projected/6f411fd9-d361-42cf-8268-d17b6c94dff5-kube-api-access-6vfzs\") pod \"multus-additional-cni-plugins-4bxmf\" (UID: \"6f411fd9-d361-42cf-8268-d17b6c94dff5\") " pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:19.883236 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.883186    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clnq6\" (UniqueName: \"kubernetes.io/projected/cae07d88-4fe4-41c8-b2f9-2b8abb3bc353-kube-api-access-clnq6\") pod \"node-ca-gp9kv\" (UID: \"cae07d88-4fe4-41c8-b2f9-2b8abb3bc353\") " pod="openshift-image-registry/node-ca-gp9kv"
Sep 14 02:52:19.883327 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.883289    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9t2k\" (UniqueName: \"kubernetes.io/projected/e343d514-241a-41a4-86a6-bbdb05394f79-kube-api-access-g9t2k\") pod \"aws-ebs-csi-driver-node-lxgqc\" (UID: \"e343d514-241a-41a4-86a6-bbdb05394f79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.883585 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.883564    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnqhb\" (UniqueName: \"kubernetes.io/projected/35e90f11-d246-46e8-aa8b-ca9448f164e3-kube-api-access-jnqhb\") pod \"network-metrics-daemon-ct7gq\" (UID: \"35e90f11-d246-46e8-aa8b-ca9448f164e3\") " pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:19.941152 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.941126    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mhwbd"
Sep 14 02:52:19.952668 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.952644    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q9w7g"
Sep 14 02:52:19.964613 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.964594    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f6w6v"
Sep 14 02:52:19.970099 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.970078    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:19.976076 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.976057    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r2jvl"
Sep 14 02:52:19.982404 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.982384    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc"
Sep 14 02:52:19.990209 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.990166    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gp9kv"
Sep 14 02:52:19.996895 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:19.996877    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6wg97"
Sep 14 02:52:20.003544 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.003527    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4bxmf"
Sep 14 02:52:20.007427 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.007412    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 02:52:20.075010 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.074990    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 02:52:20.262411 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.262327    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret\") pod \"global-pull-secret-syncer-h5wrd\" (UID: \"2407936d-8e27-4f88-93a8-9dca1a2b80b9\") " pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:20.262587 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:20.262471    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:52:20.262587 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:20.262543    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret podName:2407936d-8e27-4f88-93a8-9dca1a2b80b9 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:21.26252489 +0000 UTC m=+4.048586783 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret") pod "global-pull-secret-syncer-h5wrd" (UID: "2407936d-8e27-4f88-93a8-9dca1a2b80b9") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:52:20.363300 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.363265    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4wtg\" (UniqueName: \"kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg\") pod \"network-check-target-484dv\" (UID: \"fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4\") " pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:20.363473 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.363340    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs\") pod \"network-metrics-daemon-ct7gq\" (UID: \"35e90f11-d246-46e8-aa8b-ca9448f164e3\") " pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:20.363473 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:20.363463    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:52:20.363562 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:20.363483    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:52:20.363562 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:20.363488    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:52:20.363562 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:20.363507    2579 projected.go:196] Error preparing data for projected volume kube-api-access-m4wtg for pod openshift-network-diagnostics/network-check-target-484dv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:52:20.363562 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:20.363555    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs podName:35e90f11-d246-46e8-aa8b-ca9448f164e3 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:21.363533539 +0000 UTC m=+4.149595448 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs") pod "network-metrics-daemon-ct7gq" (UID: "35e90f11-d246-46e8-aa8b-ca9448f164e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:52:20.363716 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:20.363571    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg podName:fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:21.363564571 +0000 UTC m=+4.149626464 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4wtg" (UniqueName: "kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg") pod "network-check-target-484dv" (UID: "fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:52:20.705321 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:20.705289    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a65bc4c_0400_4b9f_ab60_75b0ff402476.slice/crio-8db8bc6b1dbb910edfb54fc365c737dfdf86b5046a5b8eb673c561865b640477 WatchSource:0}: Error finding container 8db8bc6b1dbb910edfb54fc365c737dfdf86b5046a5b8eb673c561865b640477: Status 404 returned error can't find the container with id 8db8bc6b1dbb910edfb54fc365c737dfdf86b5046a5b8eb673c561865b640477
Sep 14 02:52:20.706181 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:20.706158    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod278ffea9_7be2_48a9_b480_451029260897.slice/crio-642a9946a0271e8a34e7b8ae742d817e56a5369fcd7f4d94b3860100138a0228 WatchSource:0}: Error finding container 642a9946a0271e8a34e7b8ae742d817e56a5369fcd7f4d94b3860100138a0228: Status 404 returned error can't find the container with id 642a9946a0271e8a34e7b8ae742d817e56a5369fcd7f4d94b3860100138a0228
Sep 14 02:52:20.707049 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.707016    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 02:47:20 +0000 UTC" deadline="2028-04-14 19:16:03.955139439 +0000 UTC"
Sep 14 02:52:20.707049 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.707048    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13888h23m43.248096331s"
Sep 14 02:52:20.707547 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:20.707513    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a6b9f0_9453_482b_aa52_a8526bdfcf15.slice/crio-4055b5fe5e2d985d1690cbae1e9fbda10530ebf0308ab95e2c8d4fb3e37040b6 WatchSource:0}: Error finding container 4055b5fe5e2d985d1690cbae1e9fbda10530ebf0308ab95e2c8d4fb3e37040b6: Status 404 returned error can't find the container with id 4055b5fe5e2d985d1690cbae1e9fbda10530ebf0308ab95e2c8d4fb3e37040b6
Sep 14 02:52:20.710869 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:20.710840    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45b773f0_c1aa_456b_b7e0_9427682f4bd9.slice/crio-41134397cdc1ea71b26e28b1f4336b30e9008e4833ecbd549b0b3ae0f3b4bc9e WatchSource:0}: Error finding container 41134397cdc1ea71b26e28b1f4336b30e9008e4833ecbd549b0b3ae0f3b4bc9e: Status 404 returned error can't find the container with id 41134397cdc1ea71b26e28b1f4336b30e9008e4833ecbd549b0b3ae0f3b4bc9e
Sep 14 02:52:20.716324 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:20.716303    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode343d514_241a_41a4_86a6_bbdb05394f79.slice/crio-a902f569d304f6d00c5dff22a808701b66c96b50bad0ba4ca1489620f77df549 WatchSource:0}: Error finding container a902f569d304f6d00c5dff22a808701b66c96b50bad0ba4ca1489620f77df549: Status 404 returned error can't find the container with id a902f569d304f6d00c5dff22a808701b66c96b50bad0ba4ca1489620f77df549
Sep 14 02:52:20.717537 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:20.717511    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae07d88_4fe4_41c8_b2f9_2b8abb3bc353.slice/crio-ec4ec206bddaaf4889840db0ce8d722b16259e0ea1334ef28a3955a5a2cceeb4 WatchSource:0}: Error finding container ec4ec206bddaaf4889840db0ce8d722b16259e0ea1334ef28a3955a5a2cceeb4: Status 404 returned error can't find the container with id ec4ec206bddaaf4889840db0ce8d722b16259e0ea1334ef28a3955a5a2cceeb4
Sep 14 02:52:20.717947 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:20.717917    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3311ebca_8cba_4235_b0bd_a9908b50c143.slice/crio-30a52d915a3a1941ea1142077de17bc04b3e832ce535dc06faa1f9a2c5101b16 WatchSource:0}: Error finding container 30a52d915a3a1941ea1142077de17bc04b3e832ce535dc06faa1f9a2c5101b16: Status 404 returned error can't find the container with id 30a52d915a3a1941ea1142077de17bc04b3e832ce535dc06faa1f9a2c5101b16
Sep 14 02:52:20.719094 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:20.718980    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f411fd9_d361_42cf_8268_d17b6c94dff5.slice/crio-1c881dfcebd7ab4d3cc1583058d2c30d6d536f339f00c6ee8f25821e246f7b4e WatchSource:0}: Error finding container 1c881dfcebd7ab4d3cc1583058d2c30d6d536f339f00c6ee8f25821e246f7b4e: Status 404 returned error can't find the container with id 1c881dfcebd7ab4d3cc1583058d2c30d6d536f339f00c6ee8f25821e246f7b4e
Sep 14 02:52:20.720427 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:20.720405    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6897cf2_5e8d_49f9_a31d_a408a10acbed.slice/crio-e7a828a239d57affd6c092c16c9e17edd2041fe019170bd6396e57994e2b8c90 WatchSource:0}: Error finding container e7a828a239d57affd6c092c16c9e17edd2041fe019170bd6396e57994e2b8c90: Status 404 returned error can't find the container with id e7a828a239d57affd6c092c16c9e17edd2041fe019170bd6396e57994e2b8c90
Sep 14 02:52:20.738476 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.738065    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:20.738476 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:20.738180    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:20.745366 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.745336    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6wg97" event={"ID":"f0a6b9f0-9453-482b-aa52-a8526bdfcf15","Type":"ContainerStarted","Data":"4055b5fe5e2d985d1690cbae1e9fbda10530ebf0308ab95e2c8d4fb3e37040b6"}
Sep 14 02:52:20.747355 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.747331    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gp9kv" event={"ID":"cae07d88-4fe4-41c8-b2f9-2b8abb3bc353","Type":"ContainerStarted","Data":"ec4ec206bddaaf4889840db0ce8d722b16259e0ea1334ef28a3955a5a2cceeb4"}
Sep 14 02:52:20.751819 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.751795    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q9w7g" event={"ID":"45b773f0-c1aa-456b-b7e0-9427682f4bd9","Type":"ContainerStarted","Data":"41134397cdc1ea71b26e28b1f4336b30e9008e4833ecbd549b0b3ae0f3b4bc9e"}
Sep 14 02:52:20.756957 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.756936    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r2jvl" event={"ID":"278ffea9-7be2-48a9-b480-451029260897","Type":"ContainerStarted","Data":"642a9946a0271e8a34e7b8ae742d817e56a5369fcd7f4d94b3860100138a0228"}
Sep 14 02:52:20.758532 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.758512    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mhwbd" event={"ID":"1a65bc4c-0400-4b9f-ab60-75b0ff402476","Type":"ContainerStarted","Data":"8db8bc6b1dbb910edfb54fc365c737dfdf86b5046a5b8eb673c561865b640477"}
Sep 14 02:52:20.759720 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.759697    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb" event={"ID":"c6897cf2-5e8d-49f9-a31d-a408a10acbed","Type":"ContainerStarted","Data":"e7a828a239d57affd6c092c16c9e17edd2041fe019170bd6396e57994e2b8c90"}
Sep 14 02:52:20.761115 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.761096    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc" event={"ID":"e343d514-241a-41a4-86a6-bbdb05394f79","Type":"ContainerStarted","Data":"a902f569d304f6d00c5dff22a808701b66c96b50bad0ba4ca1489620f77df549"}
Sep 14 02:52:20.762210 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.762157    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bxmf" event={"ID":"6f411fd9-d361-42cf-8268-d17b6c94dff5","Type":"ContainerStarted","Data":"1c881dfcebd7ab4d3cc1583058d2c30d6d536f339f00c6ee8f25821e246f7b4e"}
Sep 14 02:52:20.766564 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:20.766540    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f6w6v" event={"ID":"3311ebca-8cba-4235-b0bd-a9908b50c143","Type":"ContainerStarted","Data":"30a52d915a3a1941ea1142077de17bc04b3e832ce535dc06faa1f9a2c5101b16"}
Sep 14 02:52:21.272310 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:21.272005    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret\") pod \"global-pull-secret-syncer-h5wrd\" (UID: \"2407936d-8e27-4f88-93a8-9dca1a2b80b9\") " pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:21.272464 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:21.272305    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:52:21.272464 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:21.272418    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret podName:2407936d-8e27-4f88-93a8-9dca1a2b80b9 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:23.272395315 +0000 UTC m=+6.058457225 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret") pod "global-pull-secret-syncer-h5wrd" (UID: "2407936d-8e27-4f88-93a8-9dca1a2b80b9") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:52:21.372955 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:21.372913    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4wtg\" (UniqueName: \"kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg\") pod \"network-check-target-484dv\" (UID: \"fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4\") " pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:21.373143 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:21.372990    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs\") pod \"network-metrics-daemon-ct7gq\" (UID: \"35e90f11-d246-46e8-aa8b-ca9448f164e3\") " pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:21.373143 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:21.373130    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:52:21.373307 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:21.373197    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs podName:35e90f11-d246-46e8-aa8b-ca9448f164e3 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:23.373176655 +0000 UTC m=+6.159238554 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs") pod "network-metrics-daemon-ct7gq" (UID: "35e90f11-d246-46e8-aa8b-ca9448f164e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:52:21.373679 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:21.373654    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:52:21.373809 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:21.373685    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:52:21.373809 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:21.373698    2579 projected.go:196] Error preparing data for projected volume kube-api-access-m4wtg for pod openshift-network-diagnostics/network-check-target-484dv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:52:21.373809 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:21.373747    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg podName:fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:23.373731073 +0000 UTC m=+6.159792966 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4wtg" (UniqueName: "kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg") pod "network-check-target-484dv" (UID: "fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:52:21.709167 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:21.709017    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 02:47:20 +0000 UTC" deadline="2028-03-05 12:58:33.051684618 +0000 UTC"
Sep 14 02:52:21.709167 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:21.709073    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12922h6m11.342616438s"
Sep 14 02:52:21.754045 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:21.754012    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:21.754191 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:21.754165    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:21.754613 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:21.754589    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:21.754717 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:21.754698    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:21.796011 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:21.795490    2579 generic.go:356] "Generic (PLEG): container finished" podID="ffe75fa59184a13c6e136c98cae952e6" containerID="b815c0c5126bceddaa44722e807977129ff631aaae9111edfabd1c381996ae7e" exitCode=0
Sep 14 02:52:21.796011 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:21.795573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal" event={"ID":"ffe75fa59184a13c6e136c98cae952e6","Type":"ContainerDied","Data":"b815c0c5126bceddaa44722e807977129ff631aaae9111edfabd1c381996ae7e"}
Sep 14 02:52:21.816233 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:21.816198    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal" event={"ID":"be05b9bf67903a6091aac71677a6fb10","Type":"ContainerStarted","Data":"0e4a634f805994c029f2682b917dd557265d26d57b5e7a633604c68e09748b91"}
Sep 14 02:52:22.738327 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:22.738282    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:22.738841 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:22.738426    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:22.869344 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:22.868747    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal" event={"ID":"ffe75fa59184a13c6e136c98cae952e6","Type":"ContainerStarted","Data":"1c3ae1b1fc3afd791680898ddf98e0985ead483bf6812920c56bf6ffd3fd9998"}
Sep 14 02:52:22.886724 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:22.886665    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-244.ec2.internal" podStartSLOduration=3.88664182 podStartE2EDuration="3.88664182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:52:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:21.838779546 +0000 UTC m=+4.624841465" watchObservedRunningTime="2026-09-14 02:52:22.88664182 +0000 UTC m=+5.672703737"
Sep 14 02:52:22.886961 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:22.886929    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-244.ec2.internal" podStartSLOduration=3.886919061 podStartE2EDuration="3.886919061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:52:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:22.886390664 +0000 UTC m=+5.672452583" watchObservedRunningTime="2026-09-14 02:52:22.886919061 +0000 UTC m=+5.672980979"
Sep 14 02:52:23.287642 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:23.287601    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret\") pod \"global-pull-secret-syncer-h5wrd\" (UID: \"2407936d-8e27-4f88-93a8-9dca1a2b80b9\") " pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:23.287853 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:23.287833    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:52:23.287940 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:23.287914    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret podName:2407936d-8e27-4f88-93a8-9dca1a2b80b9 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:27.287894176 +0000 UTC m=+10.073956094 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret") pod "global-pull-secret-syncer-h5wrd" (UID: "2407936d-8e27-4f88-93a8-9dca1a2b80b9") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:52:23.388901 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:23.388839    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4wtg\" (UniqueName: \"kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg\") pod \"network-check-target-484dv\" (UID: \"fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4\") " pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:23.389072 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:23.388912    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs\") pod \"network-metrics-daemon-ct7gq\" (UID: \"35e90f11-d246-46e8-aa8b-ca9448f164e3\") " pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:23.389072 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:23.389064    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:52:23.389204 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:23.389130    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs podName:35e90f11-d246-46e8-aa8b-ca9448f164e3 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:27.389110164 +0000 UTC m=+10.175172083 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs") pod "network-metrics-daemon-ct7gq" (UID: "35e90f11-d246-46e8-aa8b-ca9448f164e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:52:23.389629 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:23.389605    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:52:23.389733 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:23.389640    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:52:23.389733 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:23.389663    2579 projected.go:196] Error preparing data for projected volume kube-api-access-m4wtg for pod openshift-network-diagnostics/network-check-target-484dv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:52:23.389733 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:23.389712    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg podName:fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:27.389696742 +0000 UTC m=+10.175758652 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4wtg" (UniqueName: "kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg") pod "network-check-target-484dv" (UID: "fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:52:23.737952 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:23.737867    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:23.738122 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:23.738028    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:23.738477 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:23.738452    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:23.738863 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:23.738580    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:24.738071 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:24.738031    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:24.738301 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:24.738183    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:25.738568 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:25.737994    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:25.738568 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:25.738144    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:25.738568 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:25.738546    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:25.739173 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:25.738654    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:26.737642 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:26.737610    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:26.737838 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:26.737746    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:27.322248 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:27.322169    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret\") pod \"global-pull-secret-syncer-h5wrd\" (UID: \"2407936d-8e27-4f88-93a8-9dca1a2b80b9\") " pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:27.322651 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:27.322374    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:52:27.322651 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:27.322454    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret podName:2407936d-8e27-4f88-93a8-9dca1a2b80b9 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:35.322433319 +0000 UTC m=+18.108495228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret") pod "global-pull-secret-syncer-h5wrd" (UID: "2407936d-8e27-4f88-93a8-9dca1a2b80b9") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:52:27.423095 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:27.423061    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4wtg\" (UniqueName: \"kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg\") pod \"network-check-target-484dv\" (UID: \"fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4\") " pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:27.423313 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:27.423108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs\") pod \"network-metrics-daemon-ct7gq\" (UID: \"35e90f11-d246-46e8-aa8b-ca9448f164e3\") " pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:27.423313 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:27.423257    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:52:27.423313 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:27.423282    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:52:27.423313 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:27.423308    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:52:27.423485 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:27.423324    2579 projected.go:196] Error preparing data for projected volume kube-api-access-m4wtg for pod openshift-network-diagnostics/network-check-target-484dv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:52:27.423485 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:27.423328    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs podName:35e90f11-d246-46e8-aa8b-ca9448f164e3 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:35.423307499 +0000 UTC m=+18.209369408 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs") pod "network-metrics-daemon-ct7gq" (UID: "35e90f11-d246-46e8-aa8b-ca9448f164e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:52:27.423485 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:27.423380    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg podName:fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:35.423362254 +0000 UTC m=+18.209424154 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4wtg" (UniqueName: "kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg") pod "network-check-target-484dv" (UID: "fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:52:27.742070 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:27.742001    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:27.742238 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:27.742122    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:27.742313 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:27.742233    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:27.742368 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:27.742353    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:28.737495 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:28.737460    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:28.737929 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:28.737601    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:29.741166 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:29.740628    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:29.741166 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:29.740736    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:29.741166 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:29.740628    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:29.741166 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:29.741094    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:30.738160 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:30.738128    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:30.738354 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:30.738268    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:31.737958 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:31.737922    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:31.738350 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:31.738013    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:31.738350 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:31.738050    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:31.738350 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:31.738106    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:32.737487 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:32.737453    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:32.737661 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:32.737566    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:33.738219 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:33.737993    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:33.738219 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:33.738000    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:33.738219 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:33.738154    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:33.738219 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:33.738222    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:34.737546 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:34.737508    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:34.737767 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:34.737619    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:35.379582 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:35.379551    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret\") pod \"global-pull-secret-syncer-h5wrd\" (UID: \"2407936d-8e27-4f88-93a8-9dca1a2b80b9\") " pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:35.380122 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:35.379716    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:52:35.380122 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:35.379787    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret podName:2407936d-8e27-4f88-93a8-9dca1a2b80b9 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:51.379768091 +0000 UTC m=+34.165829986 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret") pod "global-pull-secret-syncer-h5wrd" (UID: "2407936d-8e27-4f88-93a8-9dca1a2b80b9") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:52:35.480536 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:35.480508    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4wtg\" (UniqueName: \"kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg\") pod \"network-check-target-484dv\" (UID: \"fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4\") " pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:35.480688 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:35.480550    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs\") pod \"network-metrics-daemon-ct7gq\" (UID: \"35e90f11-d246-46e8-aa8b-ca9448f164e3\") " pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:35.480688 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:35.480656    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:52:35.480763 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:35.480699    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:52:35.480763 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:35.480719    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:52:35.480763 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:35.480723    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs podName:35e90f11-d246-46e8-aa8b-ca9448f164e3 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:51.48070304 +0000 UTC m=+34.266764949 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs") pod "network-metrics-daemon-ct7gq" (UID: "35e90f11-d246-46e8-aa8b-ca9448f164e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:52:35.480763 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:35.480731    2579 projected.go:196] Error preparing data for projected volume kube-api-access-m4wtg for pod openshift-network-diagnostics/network-check-target-484dv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:52:35.480763 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:35.480764    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg podName:fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:51.480754557 +0000 UTC m=+34.266816450 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4wtg" (UniqueName: "kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg") pod "network-check-target-484dv" (UID: "fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:52:35.737790 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:35.737765    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:35.737882 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:35.737776    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:35.737919 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:35.737880    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:35.738045 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:35.738023    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:36.737943 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:36.737910    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:36.738378 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:36.738007    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:37.738764 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:37.738733    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:37.739225 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:37.738854    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:37.739225 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:37.738933    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:37.739225 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:37.739008    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:38.737451 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:38.737421    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:38.737625 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:38.737532    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:39.738294 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.738068    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:39.738721 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.738091    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:39.738721 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:39.738426    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:39.738721 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:39.738474    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:39.906566 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.906543    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gp9kv" event={"ID":"cae07d88-4fe4-41c8-b2f9-2b8abb3bc353","Type":"ContainerStarted","Data":"d6067285aa692c6646ff6c3166837b3056ad9f682c1fbff8d2f78633dbcecd16"}
Sep 14 02:52:39.907845 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.907826    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q9w7g" event={"ID":"45b773f0-c1aa-456b-b7e0-9427682f4bd9","Type":"ContainerStarted","Data":"5301b938f0c85a1bf0215c6668bb488091ae29996f8380f70d4c368ce4341ff8"}
Sep 14 02:52:39.912592 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.912564    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r2jvl" event={"ID":"278ffea9-7be2-48a9-b480-451029260897","Type":"ContainerStarted","Data":"e4fd46dfcc7525f1ee40bc609ff872f095d41b7a428716740605f03125f58398"}
Sep 14 02:52:39.914076 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.914051    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mhwbd" event={"ID":"1a65bc4c-0400-4b9f-ab60-75b0ff402476","Type":"ContainerStarted","Data":"9c9dc8b276948d35d8c7595bd8109b29be3496381515ff7a82851f7e8cef4108"}
Sep 14 02:52:39.916220 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.916199    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb" event={"ID":"c6897cf2-5e8d-49f9-a31d-a408a10acbed","Type":"ContainerStarted","Data":"5e3b9dbb7da781df5a3770f5d502b7c19742f210e8dddf5e9491886f9185eea8"}
Sep 14 02:52:39.916318 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.916266    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb" event={"ID":"c6897cf2-5e8d-49f9-a31d-a408a10acbed","Type":"ContainerStarted","Data":"aa1f8ac7ece4255b1d7a347636ed6a85a1ecb4e018308df8d06b65e21091fcc7"}
Sep 14 02:52:39.916377 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.916317    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb" event={"ID":"c6897cf2-5e8d-49f9-a31d-a408a10acbed","Type":"ContainerStarted","Data":"85954ae39df7be0644a3673eeeb4381b0bb74d74848b83ad361849d8a7eb3a6d"}
Sep 14 02:52:39.917647 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.917628    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc" event={"ID":"e343d514-241a-41a4-86a6-bbdb05394f79","Type":"ContainerStarted","Data":"1f7cec986896854ec805d43dff0c6f914a78af5b855dbc252998c0a54e59fd6e"}
Sep 14 02:52:39.919036 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.919015    2579 generic.go:356] "Generic (PLEG): container finished" podID="6f411fd9-d361-42cf-8268-d17b6c94dff5" containerID="93027e2dd5ae2c9c3ed98867ad2c43f2bbd7a59aafe686cac5ee4a68fc42b7fe" exitCode=0
Sep 14 02:52:39.919121 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.919098    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bxmf" event={"ID":"6f411fd9-d361-42cf-8268-d17b6c94dff5","Type":"ContainerDied","Data":"93027e2dd5ae2c9c3ed98867ad2c43f2bbd7a59aafe686cac5ee4a68fc42b7fe"}
Sep 14 02:52:39.920640 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.920596    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6wg97" event={"ID":"f0a6b9f0-9453-482b-aa52-a8526bdfcf15","Type":"ContainerStarted","Data":"ad560f69f149a37c19eae347b141ec6026c3e8d32421e4773395d997fa4a524a"}
Sep 14 02:52:39.930717 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.930682    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gp9kv" podStartSLOduration=3.243158426 podStartE2EDuration="21.930672469s" totalImagesPullingTime="18.687514043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:52:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:39.930018358 +0000 UTC m=+22.716080271" watchObservedRunningTime="2026-09-14 02:52:39.930672469 +0000 UTC m=+22.716734381"
Sep 14 02:52:39.965784 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.965748    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mhwbd" podStartSLOduration=3.327294651 podStartE2EDuration="21.965735408s" totalImagesPullingTime="18.638440757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:52:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:39.965564422 +0000 UTC m=+22.751626346" watchObservedRunningTime="2026-09-14 02:52:39.965735408 +0000 UTC m=+22.751797320"
Sep 14 02:52:39.982237 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.982198    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-q9w7g" podStartSLOduration=3.280963964 podStartE2EDuration="21.982168511s" totalImagesPullingTime="18.701204547s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:52:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:39.981965542 +0000 UTC m=+22.768027449" watchObservedRunningTime="2026-09-14 02:52:39.982168511 +0000 UTC m=+22.768230434"
Sep 14 02:52:40.000032 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:39.999999    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-r2jvl" podStartSLOduration=3.300697516 podStartE2EDuration="21.99998933s" totalImagesPullingTime="18.699291814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:52:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:39.999822962 +0000 UTC m=+22.785884874" watchObservedRunningTime="2026-09-14 02:52:39.99998933 +0000 UTC m=+22.786051242"
Sep 14 02:52:40.018472 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:40.018323    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6wg97" podStartSLOduration=3.383029527 podStartE2EDuration="22.018311608s" totalImagesPullingTime="18.635282081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:40.017952491 +0000 UTC m=+22.804014403" watchObservedRunningTime="2026-09-14 02:52:40.018311608 +0000 UTC m=+22.804373521"
Sep 14 02:52:40.737969 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:40.737940    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:40.738208 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:40.738035    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:40.889551 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:40.889531    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 14 02:52:40.926459 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:40.926428    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb" event={"ID":"c6897cf2-5e8d-49f9-a31d-a408a10acbed","Type":"ContainerStarted","Data":"6d3c847176ea497f111a8dd2fd3707d58096d941ae36b03915dd8ca1955598c2"}
Sep 14 02:52:40.926579 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:40.926516    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb" event={"ID":"c6897cf2-5e8d-49f9-a31d-a408a10acbed","Type":"ContainerStarted","Data":"0e8188d6e812bf488f6b01e0d3b175a92d4320d4991635a1f8333cd3564588a4"}
Sep 14 02:52:40.926579 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:40.926569    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb" event={"ID":"c6897cf2-5e8d-49f9-a31d-a408a10acbed","Type":"ContainerStarted","Data":"1fa836efbfe0133779eddd6202a8411d4034f0a8893ab49b2a01326ddfc1e732"}
Sep 14 02:52:40.928711 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:40.928689    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc" event={"ID":"e343d514-241a-41a4-86a6-bbdb05394f79","Type":"ContainerStarted","Data":"7cf271023367afff0673de9d89dd790bc23028fa5874540b99edf46a9269cdec"}
Sep 14 02:52:40.930237 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:40.930210    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f6w6v" event={"ID":"3311ebca-8cba-4235-b0bd-a9908b50c143","Type":"ContainerStarted","Data":"749281e7bdc8f2d9a4f73e92867758781c2e62cfcc4445fafd3141add5afca97"}
Sep 14 02:52:40.949373 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:40.949328    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f6w6v" podStartSLOduration=4.256869762 podStartE2EDuration="22.949312003s" totalImagesPullingTime="18.692442241s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:52:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:40.949088394 +0000 UTC m=+23.735150308" watchObservedRunningTime="2026-09-14 02:52:40.949312003 +0000 UTC m=+23.735373915"
Sep 14 02:52:41.207244 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:41.207214    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6wg97"
Sep 14 02:52:41.207841 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:41.207812    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6wg97"
Sep 14 02:52:41.712077 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:41.711976    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-14T02:52:40.889547913Z","UUID":"14876a31-87d6-4039-9ae8-6cc12f78cb15","Handler":null,"Name":"","Endpoint":""}
Sep 14 02:52:41.713704 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:41.713681    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 14 02:52:41.713847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:41.713717    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 14 02:52:41.737992 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:41.737966    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:41.737992 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:41.737981    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:41.738151 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:41.738067    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:41.738255 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:41.738234    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:41.932953 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:41.932920    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6wg97"
Sep 14 02:52:41.933366 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:41.933336    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6wg97"
Sep 14 02:52:42.737325 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:42.737243    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:42.737486 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:42.737367    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:42.937355 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:42.937319    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb" event={"ID":"c6897cf2-5e8d-49f9-a31d-a408a10acbed","Type":"ContainerStarted","Data":"8f1dedc7243b90617a478b634edae4681cd866ad6b5831c3d643c57a8e154fa1"}
Sep 14 02:52:42.939148 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:42.939123    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc" event={"ID":"e343d514-241a-41a4-86a6-bbdb05394f79","Type":"ContainerStarted","Data":"2778b8d844b0fe4df5af830073020926fe491ac5dfcdeb2723bc13ee453b5988"}
Sep 14 02:52:43.738191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:43.738154    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:43.738191 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:43.738185    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:43.738418 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:43.738279    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:43.738418 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:43.738405    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:44.737969 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:44.737645    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:44.737969 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:44.737869    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:44.946924 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:44.946848    2579 generic.go:356] "Generic (PLEG): container finished" podID="6f411fd9-d361-42cf-8268-d17b6c94dff5" containerID="4abc295763d17aae19603fc2266e5d47d3779664bb9d0946de5074f6719749e1" exitCode=0
Sep 14 02:52:44.947069 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:44.946927    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bxmf" event={"ID":"6f411fd9-d361-42cf-8268-d17b6c94dff5","Type":"ContainerDied","Data":"4abc295763d17aae19603fc2266e5d47d3779664bb9d0946de5074f6719749e1"}
Sep 14 02:52:44.950712 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:44.950688    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb" event={"ID":"c6897cf2-5e8d-49f9-a31d-a408a10acbed","Type":"ContainerStarted","Data":"ada9edf3404fd6427b5a5bccd3efb7907f8a2fa5951cd72f2f7866414ba0a401"}
Sep 14 02:52:44.950929 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:44.950907    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:44.966564 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:44.966542    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:44.973999 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:44.973961    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lxgqc" podStartSLOduration=6.247501853 podStartE2EDuration="26.973948617s" totalImagesPullingTime="20.726446764s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:52:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:42.970895959 +0000 UTC m=+25.756957873" watchObservedRunningTime="2026-09-14 02:52:44.973948617 +0000 UTC m=+27.760010530"
Sep 14 02:52:45.010016 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:45.009977    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb" podStartSLOduration=8.101928135 podStartE2EDuration="27.009962997s" totalImagesPullingTime="18.908034862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:52:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:45.00761668 +0000 UTC m=+27.793678587" watchObservedRunningTime="2026-09-14 02:52:45.009962997 +0000 UTC m=+27.796024939"
Sep 14 02:52:45.737493 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:45.737471    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:45.737566 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:45.737494    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:45.737634 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:45.737584    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:45.737706 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:45.737686    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:45.955615 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:45.955529    2579 generic.go:356] "Generic (PLEG): container finished" podID="6f411fd9-d361-42cf-8268-d17b6c94dff5" containerID="9e72ff07cc882f02a4cc15b5e5342e4428492c9f7546ee6fc368c4fa9c3071a2" exitCode=0
Sep 14 02:52:45.957043 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:45.957012    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bxmf" event={"ID":"6f411fd9-d361-42cf-8268-d17b6c94dff5","Type":"ContainerDied","Data":"9e72ff07cc882f02a4cc15b5e5342e4428492c9f7546ee6fc368c4fa9c3071a2"}
Sep 14 02:52:45.957200 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:45.957077    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:45.957282 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:45.957267    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:45.975843 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:45.975822    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:52:46.691615 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:46.691435    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ct7gq"]
Sep 14 02:52:46.691759 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:46.691744    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:46.691864 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:46.691845    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:46.692194 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:46.692152    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h5wrd"]
Sep 14 02:52:46.692279 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:46.692267    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:46.692375 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:46.692353    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:46.693855 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:46.693826    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-484dv"]
Sep 14 02:52:46.693955 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:46.693930    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:46.694036 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:46.694017    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:46.960315 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:46.960287    2579 generic.go:356] "Generic (PLEG): container finished" podID="6f411fd9-d361-42cf-8268-d17b6c94dff5" containerID="8b9e7d8e69e5c239a5ccaa34adb5612bab441454696685c87c276d5306f55812" exitCode=0
Sep 14 02:52:46.960773 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:46.960371    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bxmf" event={"ID":"6f411fd9-d361-42cf-8268-d17b6c94dff5","Type":"ContainerDied","Data":"8b9e7d8e69e5c239a5ccaa34adb5612bab441454696685c87c276d5306f55812"}
Sep 14 02:52:47.862703 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:47.862670    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wg97_f0a6b9f0-9453-482b-aa52-a8526bdfcf15/konnectivity-agent/0.log" limit=1024
Sep 14 02:52:47.984739 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:47.984704    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wg97_f0a6b9f0-9453-482b-aa52-a8526bdfcf15/konnectivity-agent/0.log" limit=1024
Sep 14 02:52:48.737944 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:48.737881    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:48.737944 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:48.737924    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:48.738090 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:48.737993    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:48.738090 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:48.737926    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:48.738090 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:48.738034    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:48.738218 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:48.738109    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:50.737847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:50.737809    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:50.738289 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:50.737857    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:50.738289 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:50.737918    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:50.738289 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:50.737934    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-484dv" podUID="fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4"
Sep 14 02:52:50.738289 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:50.738007    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ct7gq" podUID="35e90f11-d246-46e8-aa8b-ca9448f164e3"
Sep 14 02:52:50.738289 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:50.738086    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h5wrd" podUID="2407936d-8e27-4f88-93a8-9dca1a2b80b9"
Sep 14 02:52:51.395982 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:51.395959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret\") pod \"global-pull-secret-syncer-h5wrd\" (UID: \"2407936d-8e27-4f88-93a8-9dca1a2b80b9\") " pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:51.396115 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:51.396098    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 02:52:51.396166 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:51.396157    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret podName:2407936d-8e27-4f88-93a8-9dca1a2b80b9 nodeName:}" failed. No retries permitted until 2026-09-14 02:53:23.396143084 +0000 UTC m=+66.182204978 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret") pod "global-pull-secret-syncer-h5wrd" (UID: "2407936d-8e27-4f88-93a8-9dca1a2b80b9") : object "kube-system"/"original-pull-secret" not registered
Sep 14 02:52:51.496699 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:51.496546    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4wtg\" (UniqueName: \"kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg\") pod \"network-check-target-484dv\" (UID: \"fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4\") " pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:51.496813 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:51.496705    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 02:52:51.496813 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:51.496725    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 02:52:51.496813 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:51.496738    2579 projected.go:196] Error preparing data for projected volume kube-api-access-m4wtg for pod openshift-network-diagnostics/network-check-target-484dv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:52:51.496813 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:51.496782    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg podName:fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4 nodeName:}" failed. No retries permitted until 2026-09-14 02:53:23.496766851 +0000 UTC m=+66.282828742 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4wtg" (UniqueName: "kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg") pod "network-check-target-484dv" (UID: "fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 02:52:51.496813 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:51.496789    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:52:51.496813 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:51.496723    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs\") pod \"network-metrics-daemon-ct7gq\" (UID: \"35e90f11-d246-46e8-aa8b-ca9448f164e3\") " pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:51.497003 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:51.496826    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs podName:35e90f11-d246-46e8-aa8b-ca9448f164e3 nodeName:}" failed. No retries permitted until 2026-09-14 02:53:23.496815455 +0000 UTC m=+66.282877346 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs") pod "network-metrics-daemon-ct7gq" (UID: "35e90f11-d246-46e8-aa8b-ca9448f164e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 02:52:51.974654 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:51.974625    2579 generic.go:356] "Generic (PLEG): container finished" podID="6f411fd9-d361-42cf-8268-d17b6c94dff5" containerID="c2d145602949c9249a35bf05825c0654a7cf4ff16f74bd92557ccf8529a46ec2" exitCode=0
Sep 14 02:52:51.975000 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:51.974663    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bxmf" event={"ID":"6f411fd9-d361-42cf-8268-d17b6c94dff5","Type":"ContainerDied","Data":"c2d145602949c9249a35bf05825c0654a7cf4ff16f74bd92557ccf8529a46ec2"}
Sep 14 02:52:52.545938 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.545910    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-244.ec2.internal" event="NodeReady"
Sep 14 02:52:52.546139 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.546031    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 14 02:52:52.620699 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.620671    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pq276"]
Sep 14 02:52:52.640118 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.640098    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-269h2"]
Sep 14 02:52:52.640239 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.640218    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pq276"
Sep 14 02:52:52.643026 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.643003    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gj97m\""
Sep 14 02:52:52.643698 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.643675    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 14 02:52:52.643808 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.643724    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 14 02:52:52.656143 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.656126    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pq276"]
Sep 14 02:52:52.656242 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.656155    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-269h2"]
Sep 14 02:52:52.656242 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.656232    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-269h2"
Sep 14 02:52:52.659294 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.659277    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 14 02:52:52.660086 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.660070    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 14 02:52:52.660086 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.660082    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gwhfd\""
Sep 14 02:52:52.660322 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.660311    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 14 02:52:52.737815 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.737776    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:52:52.737930 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.737887    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:52:52.737930 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.737898    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:52:52.741614 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.741596    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lx47r\""
Sep 14 02:52:52.741713 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.741630    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 02:52:52.741713 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.741705    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 02:52:52.741818 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.741768    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 02:52:52.741938 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.741922    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hn74x\""
Sep 14 02:52:52.742218 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.742205    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 02:52:52.781707 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.781682    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wg97_f0a6b9f0-9453-482b-aa52-a8526bdfcf15/konnectivity-agent/0.log" limit=1024
Sep 14 02:52:52.805730 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.805712    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-tmp-dir\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:52:52.805820 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.805739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-config-volume\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:52:52.805820 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.805775    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnhx9\" (UniqueName: \"kubernetes.io/projected/ea8c0670-0161-4347-ad46-98885e8edd2b-kube-api-access-tnhx9\") pod \"ingress-canary-269h2\" (UID: \"ea8c0670-0161-4347-ad46-98885e8edd2b\") " pod="openshift-ingress-canary/ingress-canary-269h2"
Sep 14 02:52:52.805908 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.805821    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b89k8\" (UniqueName: \"kubernetes.io/projected/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-kube-api-access-b89k8\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:52:52.805908 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.805874    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:52:52.805908 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.805902    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea8c0670-0161-4347-ad46-98885e8edd2b-cert\") pod \"ingress-canary-269h2\" (UID: \"ea8c0670-0161-4347-ad46-98885e8edd2b\") " pod="openshift-ingress-canary/ingress-canary-269h2"
Sep 14 02:52:52.881650 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.881625    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wg97_f0a6b9f0-9453-482b-aa52-a8526bdfcf15/konnectivity-agent/0.log" limit=1024
Sep 14 02:52:52.906945 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.906922    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-config-volume\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:52:52.907024 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.906977    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnhx9\" (UniqueName: \"kubernetes.io/projected/ea8c0670-0161-4347-ad46-98885e8edd2b-kube-api-access-tnhx9\") pod \"ingress-canary-269h2\" (UID: \"ea8c0670-0161-4347-ad46-98885e8edd2b\") " pod="openshift-ingress-canary/ingress-canary-269h2"
Sep 14 02:52:52.907024 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.907001    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b89k8\" (UniqueName: \"kubernetes.io/projected/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-kube-api-access-b89k8\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:52:52.907090 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.907038    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:52:52.907090 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.907060    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea8c0670-0161-4347-ad46-98885e8edd2b-cert\") pod \"ingress-canary-269h2\" (UID: \"ea8c0670-0161-4347-ad46-98885e8edd2b\") " pod="openshift-ingress-canary/ingress-canary-269h2"
Sep 14 02:52:52.907160 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.907097    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-tmp-dir\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:52:52.907231 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:52.907205    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:52:52.907286 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:52.907270    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls podName:56e1da36-b3ce-4cf4-b99c-dd9a7822c73c nodeName:}" failed. No retries permitted until 2026-09-14 02:52:53.407250331 +0000 UTC m=+36.193312239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls") pod "dns-default-pq276" (UID: "56e1da36-b3ce-4cf4-b99c-dd9a7822c73c") : secret "dns-default-metrics-tls" not found
Sep 14 02:52:52.907527 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.907511    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-tmp-dir\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:52:52.907570 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.907535    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-config-volume\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:52:52.910918 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.910899    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea8c0670-0161-4347-ad46-98885e8edd2b-cert\") pod \"ingress-canary-269h2\" (UID: \"ea8c0670-0161-4347-ad46-98885e8edd2b\") " pod="openshift-ingress-canary/ingress-canary-269h2"
Sep 14 02:52:52.916565 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.916545    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b89k8\" (UniqueName: \"kubernetes.io/projected/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-kube-api-access-b89k8\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:52:52.916911 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.916888    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnhx9\" (UniqueName: \"kubernetes.io/projected/ea8c0670-0161-4347-ad46-98885e8edd2b-kube-api-access-tnhx9\") pod \"ingress-canary-269h2\" (UID: \"ea8c0670-0161-4347-ad46-98885e8edd2b\") " pod="openshift-ingress-canary/ingress-canary-269h2"
Sep 14 02:52:52.926921 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.926903    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt"]
Sep 14 02:52:52.935369 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.935353    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt"
Sep 14 02:52:52.939333 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.939108    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 14 02:52:52.939333 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.939126    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 14 02:52:52.939333 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.939202    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 14 02:52:52.939333 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.939110    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 14 02:52:52.939333 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.939258    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qgqg4\""
Sep 14 02:52:52.940263 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.940245    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt"]
Sep 14 02:52:52.963578 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.963563    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-269h2"
Sep 14 02:52:52.980524 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.980504    2579 generic.go:356] "Generic (PLEG): container finished" podID="6f411fd9-d361-42cf-8268-d17b6c94dff5" containerID="d411258e9acfdee20dde8ab2ecc5cca97d575d7f169dbb01aac57fbdf9c4ec47" exitCode=0
Sep 14 02:52:52.980872 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:52.980558    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bxmf" event={"ID":"6f411fd9-d361-42cf-8268-d17b6c94dff5","Type":"ContainerDied","Data":"d411258e9acfdee20dde8ab2ecc5cca97d575d7f169dbb01aac57fbdf9c4ec47"}
Sep 14 02:52:53.007944 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:53.007922    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbgrm\" (UniqueName: \"kubernetes.io/projected/8c3d78f2-3b6e-41d6-b747-350c6d5b1e70-kube-api-access-bbgrm\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ncwzt\" (UID: \"8c3d78f2-3b6e-41d6-b747-350c6d5b1e70\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt"
Sep 14 02:52:53.008007 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:53.007994    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8c3d78f2-3b6e-41d6-b747-350c6d5b1e70-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ncwzt\" (UID: \"8c3d78f2-3b6e-41d6-b747-350c6d5b1e70\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt"
Sep 14 02:52:53.008048 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:53.008017    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8c3d78f2-3b6e-41d6-b747-350c6d5b1e70-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ncwzt\" (UID: \"8c3d78f2-3b6e-41d6-b747-350c6d5b1e70\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt"
Sep 14 02:52:53.108711 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:53.108594    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8c3d78f2-3b6e-41d6-b747-350c6d5b1e70-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ncwzt\" (UID: \"8c3d78f2-3b6e-41d6-b747-350c6d5b1e70\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt"
Sep 14 02:52:53.108816 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:53.108756    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8c3d78f2-3b6e-41d6-b747-350c6d5b1e70-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ncwzt\" (UID: \"8c3d78f2-3b6e-41d6-b747-350c6d5b1e70\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt"
Sep 14 02:52:53.108880 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:53.108868    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbgrm\" (UniqueName: \"kubernetes.io/projected/8c3d78f2-3b6e-41d6-b747-350c6d5b1e70-kube-api-access-bbgrm\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ncwzt\" (UID: \"8c3d78f2-3b6e-41d6-b747-350c6d5b1e70\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt"
Sep 14 02:52:53.109129 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:53.109113    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8c3d78f2-3b6e-41d6-b747-350c6d5b1e70-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ncwzt\" (UID: \"8c3d78f2-3b6e-41d6-b747-350c6d5b1e70\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt"
Sep 14 02:52:53.111187 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:53.111155    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8c3d78f2-3b6e-41d6-b747-350c6d5b1e70-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ncwzt\" (UID: \"8c3d78f2-3b6e-41d6-b747-350c6d5b1e70\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt"
Sep 14 02:52:53.118583 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:53.118566    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbgrm\" (UniqueName: \"kubernetes.io/projected/8c3d78f2-3b6e-41d6-b747-350c6d5b1e70-kube-api-access-bbgrm\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ncwzt\" (UID: \"8c3d78f2-3b6e-41d6-b747-350c6d5b1e70\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt"
Sep 14 02:52:53.244518 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:53.244491    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt"
Sep 14 02:52:53.260282 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:53.260261    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-269h2"]
Sep 14 02:52:53.263499 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:53.263476    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea8c0670_0161_4347_ad46_98885e8edd2b.slice/crio-4850625ad31960b6218f119d3e6c5a91f29946122ff20bf16ce619c8fae166e1 WatchSource:0}: Error finding container 4850625ad31960b6218f119d3e6c5a91f29946122ff20bf16ce619c8fae166e1: Status 404 returned error can't find the container with id 4850625ad31960b6218f119d3e6c5a91f29946122ff20bf16ce619c8fae166e1
Sep 14 02:52:53.410777 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:53.410746    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:52:53.410901 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:53.410872    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:52:53.410940 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:53.410923    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls podName:56e1da36-b3ce-4cf4-b99c-dd9a7822c73c nodeName:}" failed. No retries permitted until 2026-09-14 02:52:54.410909669 +0000 UTC m=+37.196971573 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls") pod "dns-default-pq276" (UID: "56e1da36-b3ce-4cf4-b99c-dd9a7822c73c") : secret "dns-default-metrics-tls" not found
Sep 14 02:52:53.521114 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:53.521052    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt"]
Sep 14 02:52:53.525023 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:53.525000    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c3d78f2_3b6e_41d6_b747_350c6d5b1e70.slice/crio-5480aa0ddaedc33f5201b6c6a2b60c4aac53faa31a30290d761c3a3522f9b82d WatchSource:0}: Error finding container 5480aa0ddaedc33f5201b6c6a2b60c4aac53faa31a30290d761c3a3522f9b82d: Status 404 returned error can't find the container with id 5480aa0ddaedc33f5201b6c6a2b60c4aac53faa31a30290d761c3a3522f9b82d
Sep 14 02:52:53.984514 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:53.984437    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt" event={"ID":"8c3d78f2-3b6e-41d6-b747-350c6d5b1e70","Type":"ContainerStarted","Data":"5480aa0ddaedc33f5201b6c6a2b60c4aac53faa31a30290d761c3a3522f9b82d"}
Sep 14 02:52:53.985691 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:53.985664    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-269h2" event={"ID":"ea8c0670-0161-4347-ad46-98885e8edd2b","Type":"ContainerStarted","Data":"4850625ad31960b6218f119d3e6c5a91f29946122ff20bf16ce619c8fae166e1"}
Sep 14 02:52:53.989500 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:53.989476    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bxmf" event={"ID":"6f411fd9-d361-42cf-8268-d17b6c94dff5","Type":"ContainerStarted","Data":"b67a98d01a2be2e8f8c77f0a4344cbb213ad27c75d6310473c3ad0b2bb11021c"}
Sep 14 02:52:54.015918 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.015858    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4bxmf" podStartSLOduration=6.983715946 podStartE2EDuration="36.015842572s" totalImagesPullingTime="29.032126626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:52:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:54.015268016 +0000 UTC m=+36.801329931" watchObservedRunningTime="2026-09-14 02:52:54.015842572 +0000 UTC m=+36.801904484"
Sep 14 02:52:54.109226 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.109195    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wg97_f0a6b9f0-9453-482b-aa52-a8526bdfcf15/konnectivity-agent/0.log" limit=1024
Sep 14 02:52:54.197192 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.197139    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wg97_f0a6b9f0-9453-482b-aa52-a8526bdfcf15/konnectivity-agent/0.log" limit=1024
Sep 14 02:52:54.310701 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.310667    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wg97_f0a6b9f0-9453-482b-aa52-a8526bdfcf15/konnectivity-agent/0.log" limit=1024
Sep 14 02:52:54.382355 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.382320    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-4x8n6"]
Sep 14 02:52:54.404853 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.404819    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-4x8n6"]
Sep 14 02:52:54.404992 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.404952    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-4x8n6"
Sep 14 02:52:54.407820 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.407795    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-mmlcr\""
Sep 14 02:52:54.418509 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.418483    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:52:54.418638 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:54.418625    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:52:54.418688 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:54.418679    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls podName:56e1da36-b3ce-4cf4-b99c-dd9a7822c73c nodeName:}" failed. No retries permitted until 2026-09-14 02:52:56.418662299 +0000 UTC m=+39.204724210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls") pod "dns-default-pq276" (UID: "56e1da36-b3ce-4cf4-b99c-dd9a7822c73c") : secret "dns-default-metrics-tls" not found
Sep 14 02:52:54.519781 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.519750    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px24t\" (UniqueName: \"kubernetes.io/projected/94345e86-d295-42fc-8450-3b6f73fc6056-kube-api-access-px24t\") pod \"network-check-source-6f8b768455-4x8n6\" (UID: \"94345e86-d295-42fc-8450-3b6f73fc6056\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-4x8n6"
Sep 14 02:52:54.538427 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.538398    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wg97_f0a6b9f0-9453-482b-aa52-a8526bdfcf15/konnectivity-agent/0.log" limit=1024
Sep 14 02:52:54.550619 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.550592    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5zdn2"]
Sep 14 02:52:54.571050 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.570990    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5zdn2"]
Sep 14 02:52:54.571163 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.571098    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:54.574496 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.574463    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 14 02:52:54.574603 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.574503    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 14 02:52:54.574732 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.574470    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 14 02:52:54.575558 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.575537    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 14 02:52:54.575558 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.575550    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ztclm\""
Sep 14 02:52:54.620659 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.620629    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px24t\" (UniqueName: \"kubernetes.io/projected/94345e86-d295-42fc-8450-3b6f73fc6056-kube-api-access-px24t\") pod \"network-check-source-6f8b768455-4x8n6\" (UID: \"94345e86-d295-42fc-8450-3b6f73fc6056\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-4x8n6"
Sep 14 02:52:54.622971 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.622920    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wg97_f0a6b9f0-9453-482b-aa52-a8526bdfcf15/konnectivity-agent/0.log" limit=1024
Sep 14 02:52:54.630421 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.630395    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px24t\" (UniqueName: \"kubernetes.io/projected/94345e86-d295-42fc-8450-3b6f73fc6056-kube-api-access-px24t\") pod \"network-check-source-6f8b768455-4x8n6\" (UID: \"94345e86-d295-42fc-8450-3b6f73fc6056\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-4x8n6"
Sep 14 02:52:54.717051 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.717017    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-4x8n6"
Sep 14 02:52:54.721489 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.721460    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:54.721613 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.721505    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a890518d-c1d1-4ca7-bb6e-45032e701414-crio-socket\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:54.721613 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.721559    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh4t9\" (UniqueName: \"kubernetes.io/projected/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-api-access-zh4t9\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:54.721613 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.721601    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:54.721762 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.721633    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a890518d-c1d1-4ca7-bb6e-45032e701414-data-volume\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:54.822700 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.822638    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:54.822805 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.822727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a890518d-c1d1-4ca7-bb6e-45032e701414-data-volume\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:54.822805 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:54.822752    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 02:52:54.822805 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.822791    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:54.822966 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:54.822833    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm podName:a890518d-c1d1-4ca7-bb6e-45032e701414 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:55.322810932 +0000 UTC m=+38.108872842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5zdn2" (UID: "a890518d-c1d1-4ca7-bb6e-45032e701414") : configmap "kube-rbac-proxy" not found
Sep 14 02:52:54.822966 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.822855    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a890518d-c1d1-4ca7-bb6e-45032e701414-crio-socket\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:54.822966 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:54.822894    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 02:52:54.822966 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.822898    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4t9\" (UniqueName: \"kubernetes.io/projected/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-api-access-zh4t9\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:54.822966 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:54.822946    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls podName:a890518d-c1d1-4ca7-bb6e-45032e701414 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:55.322930453 +0000 UTC m=+38.108992348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5zdn2" (UID: "a890518d-c1d1-4ca7-bb6e-45032e701414") : secret "insights-runtime-extractor-tls" not found
Sep 14 02:52:54.823166 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.823052    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a890518d-c1d1-4ca7-bb6e-45032e701414-data-volume\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:54.823265 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.823247    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a890518d-c1d1-4ca7-bb6e-45032e701414-crio-socket\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:54.833417 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.833393    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh4t9\" (UniqueName: \"kubernetes.io/projected/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-api-access-zh4t9\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:54.927926 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.927897    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52"]
Sep 14 02:52:54.941010 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.940983    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52"]
Sep 14 02:52:54.941124 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.941091    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52"
Sep 14 02:52:54.943942 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.943916    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 14 02:52:54.944814 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.944145    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 14 02:52:54.944814 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.944358    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 14 02:52:54.944814 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.944504    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-pj6l2\""
Sep 14 02:52:54.944814 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:54.944651    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 14 02:52:55.018895 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:55.018869    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-4x8n6"]
Sep 14 02:52:55.024287 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:55.024256    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hks2x\" (UniqueName: \"kubernetes.io/projected/a9048354-a3ee-4810-8e08-20b70a5332c2-kube-api-access-hks2x\") pod \"service-ca-operator-cf7c776b7-b4z52\" (UID: \"a9048354-a3ee-4810-8e08-20b70a5332c2\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52"
Sep 14 02:52:55.024402 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:55.024386    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9048354-a3ee-4810-8e08-20b70a5332c2-serving-cert\") pod \"service-ca-operator-cf7c776b7-b4z52\" (UID: \"a9048354-a3ee-4810-8e08-20b70a5332c2\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52"
Sep 14 02:52:55.024462 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:55.024414    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9048354-a3ee-4810-8e08-20b70a5332c2-config\") pod \"service-ca-operator-cf7c776b7-b4z52\" (UID: \"a9048354-a3ee-4810-8e08-20b70a5332c2\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52"
Sep 14 02:52:55.125070 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:55.124994    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9048354-a3ee-4810-8e08-20b70a5332c2-serving-cert\") pod \"service-ca-operator-cf7c776b7-b4z52\" (UID: \"a9048354-a3ee-4810-8e08-20b70a5332c2\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52"
Sep 14 02:52:55.125239 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:55.125071    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9048354-a3ee-4810-8e08-20b70a5332c2-config\") pod \"service-ca-operator-cf7c776b7-b4z52\" (UID: \"a9048354-a3ee-4810-8e08-20b70a5332c2\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52"
Sep 14 02:52:55.125239 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:55.125127    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hks2x\" (UniqueName: \"kubernetes.io/projected/a9048354-a3ee-4810-8e08-20b70a5332c2-kube-api-access-hks2x\") pod \"service-ca-operator-cf7c776b7-b4z52\" (UID: \"a9048354-a3ee-4810-8e08-20b70a5332c2\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52"
Sep 14 02:52:55.125975 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:55.125951    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9048354-a3ee-4810-8e08-20b70a5332c2-config\") pod \"service-ca-operator-cf7c776b7-b4z52\" (UID: \"a9048354-a3ee-4810-8e08-20b70a5332c2\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52"
Sep 14 02:52:55.127890 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:55.127871    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9048354-a3ee-4810-8e08-20b70a5332c2-serving-cert\") pod \"service-ca-operator-cf7c776b7-b4z52\" (UID: \"a9048354-a3ee-4810-8e08-20b70a5332c2\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52"
Sep 14 02:52:55.138793 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:55.138773    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hks2x\" (UniqueName: \"kubernetes.io/projected/a9048354-a3ee-4810-8e08-20b70a5332c2-kube-api-access-hks2x\") pod \"service-ca-operator-cf7c776b7-b4z52\" (UID: \"a9048354-a3ee-4810-8e08-20b70a5332c2\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52"
Sep 14 02:52:55.252383 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:55.252354    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52"
Sep 14 02:52:55.327682 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:55.327650    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:55.327838 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:55.327728    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:55.327900 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:55.327832    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 02:52:55.327900 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:55.327842    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 02:52:55.327989 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:55.327906    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm podName:a890518d-c1d1-4ca7-bb6e-45032e701414 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:56.327885299 +0000 UTC m=+39.113947209 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5zdn2" (UID: "a890518d-c1d1-4ca7-bb6e-45032e701414") : configmap "kube-rbac-proxy" not found
Sep 14 02:52:55.327989 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:55.327924    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls podName:a890518d-c1d1-4ca7-bb6e-45032e701414 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:56.3279149 +0000 UTC m=+39.113976792 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5zdn2" (UID: "a890518d-c1d1-4ca7-bb6e-45032e701414") : secret "insights-runtime-extractor-tls" not found
Sep 14 02:52:55.597899 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:55.597859    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94345e86_d295_42fc_8450_3b6f73fc6056.slice/crio-b759bed219c0a75d2f559175943d762026c03ce05fb78c9e4b47a0330084d347 WatchSource:0}: Error finding container b759bed219c0a75d2f559175943d762026c03ce05fb78c9e4b47a0330084d347: Status 404 returned error can't find the container with id b759bed219c0a75d2f559175943d762026c03ce05fb78c9e4b47a0330084d347
Sep 14 02:52:55.998680 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:55.998651    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-4x8n6" event={"ID":"94345e86-d295-42fc-8450-3b6f73fc6056","Type":"ContainerStarted","Data":"b759bed219c0a75d2f559175943d762026c03ce05fb78c9e4b47a0330084d347"}
Sep 14 02:52:56.259627 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:56.259397    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wg97_f0a6b9f0-9453-482b-aa52-a8526bdfcf15/konnectivity-agent/0.log" limit=1024
Sep 14 02:52:56.296022 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:56.296000    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mhwbd_1a65bc4c-0400-4b9f-ab60-75b0ff402476/dns-node-resolver/0.log"
Sep 14 02:52:56.335032 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:56.335003    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:56.335159 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:56.335105    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:56.335159 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:56.335124    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 02:52:56.335308 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:56.335199    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm podName:a890518d-c1d1-4ca7-bb6e-45032e701414 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:58.335165836 +0000 UTC m=+41.121227846 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5zdn2" (UID: "a890518d-c1d1-4ca7-bb6e-45032e701414") : configmap "kube-rbac-proxy" not found
Sep 14 02:52:56.335375 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:56.335327    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 02:52:56.335440 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:56.335407    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls podName:a890518d-c1d1-4ca7-bb6e-45032e701414 nodeName:}" failed. No retries permitted until 2026-09-14 02:52:58.335388064 +0000 UTC m=+41.121449982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5zdn2" (UID: "a890518d-c1d1-4ca7-bb6e-45032e701414") : secret "insights-runtime-extractor-tls" not found
Sep 14 02:52:56.436455 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:56.436416    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:52:56.436621 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:56.436570    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:52:56.436658 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:56.436633    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls podName:56e1da36-b3ce-4cf4-b99c-dd9a7822c73c nodeName:}" failed. No retries permitted until 2026-09-14 02:53:00.436617766 +0000 UTC m=+43.222679657 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls") pod "dns-default-pq276" (UID: "56e1da36-b3ce-4cf4-b99c-dd9a7822c73c") : secret "dns-default-metrics-tls" not found
Sep 14 02:52:56.527920 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:56.527850    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52"]
Sep 14 02:52:56.532534 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:52:56.532506    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9048354_a3ee_4810_8e08_20b70a5332c2.slice/crio-f248f084a42827fa8283dade67735c2d878bcef859509981836ba6a486ff9df0 WatchSource:0}: Error finding container f248f084a42827fa8283dade67735c2d878bcef859509981836ba6a486ff9df0: Status 404 returned error can't find the container with id f248f084a42827fa8283dade67735c2d878bcef859509981836ba6a486ff9df0
Sep 14 02:52:56.892023 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:56.891949    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gp9kv_cae07d88-4fe4-41c8-b2f9-2b8abb3bc353/node-ca/0.log"
Sep 14 02:52:57.003819 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:57.003784    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt" event={"ID":"8c3d78f2-3b6e-41d6-b747-350c6d5b1e70","Type":"ContainerStarted","Data":"eba09fe36157b25b3e9e040a66cd4b2a62ebf51b64bece521fa9fa7246ec625f"}
Sep 14 02:52:57.005530 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:57.005501    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-269h2" event={"ID":"ea8c0670-0161-4347-ad46-98885e8edd2b","Type":"ContainerStarted","Data":"7fd268a5283c5cdcc206bcf9246783cebe60879644e9edd3c451f692b9d0a8e5"}
Sep 14 02:52:57.006838 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:57.006805    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52" event={"ID":"a9048354-a3ee-4810-8e08-20b70a5332c2","Type":"ContainerStarted","Data":"f248f084a42827fa8283dade67735c2d878bcef859509981836ba6a486ff9df0"}
Sep 14 02:52:57.043818 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:57.043755    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt" podStartSLOduration=2.309786435 podStartE2EDuration="5.043735676s" totalImagesPullingTime="2.733949241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:57.021662482 +0000 UTC m=+39.807724394" watchObservedRunningTime="2026-09-14 02:52:57.043735676 +0000 UTC m=+39.829797590"
Sep 14 02:52:57.044733 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:57.044698    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-269h2" podStartSLOduration=2.051361231 podStartE2EDuration="5.044684571s" totalImagesPullingTime="2.99332334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:52:57.043577537 +0000 UTC m=+39.829639451" watchObservedRunningTime="2026-09-14 02:52:57.044684571 +0000 UTC m=+39.830746485"
Sep 14 02:52:57.491383 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:57.491342    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-269h2_ea8c0670-0161-4347-ad46-98885e8edd2b/serve-healthcheck-canary/0.log"
Sep 14 02:52:58.338247 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:58.338213    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-4q7sw"]
Sep 14 02:52:58.355039 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:58.354968    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:58.355204 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:58.355101    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:52:58.355293 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:58.355265    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 02:52:58.355345 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:58.355315    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm podName:a890518d-c1d1-4ca7-bb6e-45032e701414 nodeName:}" failed. No retries permitted until 2026-09-14 02:53:02.355302702 +0000 UTC m=+45.141364593 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5zdn2" (UID: "a890518d-c1d1-4ca7-bb6e-45032e701414") : configmap "kube-rbac-proxy" not found
Sep 14 02:52:58.355408 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:58.355381    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 02:52:58.355408 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:52:58.355407    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls podName:a890518d-c1d1-4ca7-bb6e-45032e701414 nodeName:}" failed. No retries permitted until 2026-09-14 02:53:02.355398529 +0000 UTC m=+45.141460420 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5zdn2" (UID: "a890518d-c1d1-4ca7-bb6e-45032e701414") : secret "insights-runtime-extractor-tls" not found
Sep 14 02:52:58.355971 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:58.355920    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-4q7sw"]
Sep 14 02:52:58.356092 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:58.356023    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4q7sw"
Sep 14 02:52:58.358856 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:58.358831    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 14 02:52:58.358965 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:58.358941    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-2d2bl\""
Sep 14 02:52:58.360010 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:58.359989    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 14 02:52:58.455609 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:58.455499    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fsbd\" (UniqueName: \"kubernetes.io/projected/fb9239b3-0ce7-4964-b4f5-b1f7547582b3-kube-api-access-5fsbd\") pod \"migrator-8b4d697c5-4q7sw\" (UID: \"fb9239b3-0ce7-4964-b4f5-b1f7547582b3\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4q7sw"
Sep 14 02:52:58.556707 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:58.556663    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fsbd\" (UniqueName: \"kubernetes.io/projected/fb9239b3-0ce7-4964-b4f5-b1f7547582b3-kube-api-access-5fsbd\") pod \"migrator-8b4d697c5-4q7sw\" (UID: \"fb9239b3-0ce7-4964-b4f5-b1f7547582b3\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4q7sw"
Sep 14 02:52:58.566025 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:58.565998    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fsbd\" (UniqueName: \"kubernetes.io/projected/fb9239b3-0ce7-4964-b4f5-b1f7547582b3-kube-api-access-5fsbd\") pod \"migrator-8b4d697c5-4q7sw\" (UID: \"fb9239b3-0ce7-4964-b4f5-b1f7547582b3\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4q7sw"
Sep 14 02:52:58.667911 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:52:58.667833    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4q7sw"
Sep 14 02:53:00.370619 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:00.370548    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-4q7sw"]
Sep 14 02:53:00.373807 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:53:00.373781    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb9239b3_0ce7_4964_b4f5_b1f7547582b3.slice/crio-5acfb2db789a136bedb56ba14c2cfe2563e697f0c9ac31ba17c5172b665456d4 WatchSource:0}: Error finding container 5acfb2db789a136bedb56ba14c2cfe2563e697f0c9ac31ba17c5172b665456d4: Status 404 returned error can't find the container with id 5acfb2db789a136bedb56ba14c2cfe2563e697f0c9ac31ba17c5172b665456d4
Sep 14 02:53:00.473241 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:00.473222    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:53:00.473410 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:00.473391    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:53:00.473473 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:00.473462    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls podName:56e1da36-b3ce-4cf4-b99c-dd9a7822c73c nodeName:}" failed. No retries permitted until 2026-09-14 02:53:08.473440473 +0000 UTC m=+51.259502370 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls") pod "dns-default-pq276" (UID: "56e1da36-b3ce-4cf4-b99c-dd9a7822c73c") : secret "dns-default-metrics-tls" not found
Sep 14 02:53:01.020509 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:01.020471    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-4x8n6" event={"ID":"94345e86-d295-42fc-8450-3b6f73fc6056","Type":"ContainerStarted","Data":"e4080e8e61e2c35bc2426d938a9830bb8d9da23bc9e3f5f657d4ffab176a47de"}
Sep 14 02:53:01.022358 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:01.022328    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52" event={"ID":"a9048354-a3ee-4810-8e08-20b70a5332c2","Type":"ContainerStarted","Data":"f3d3a0cfce0c3f5f0c5584d955ac8d3908e49a71d71bd84bb860c5fde35eb2f7"}
Sep 14 02:53:01.025138 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:01.025109    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4q7sw" event={"ID":"fb9239b3-0ce7-4964-b4f5-b1f7547582b3","Type":"ContainerStarted","Data":"5acfb2db789a136bedb56ba14c2cfe2563e697f0c9ac31ba17c5172b665456d4"}
Sep 14 02:53:01.038442 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:01.038393    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-4x8n6" podStartSLOduration=2.524178657 podStartE2EDuration="7.038379914s" totalImagesPullingTime="4.514201257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:01.037557925 +0000 UTC m=+43.823619838" watchObservedRunningTime="2026-09-14 02:53:01.038379914 +0000 UTC m=+43.824441826"
Sep 14 02:53:01.053694 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:01.053651    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52" podStartSLOduration=3.48345962 podStartE2EDuration="7.053635333s" totalImagesPullingTime="3.570175713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:01.053359726 +0000 UTC m=+43.839421641" watchObservedRunningTime="2026-09-14 02:53:01.053635333 +0000 UTC m=+43.839697244"
Sep 14 02:53:02.388904 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.388841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:53:02.388904 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.388893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:53:02.389265 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:02.388952    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 02:53:02.389265 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:02.388987    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 02:53:02.389265 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:02.388995    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm podName:a890518d-c1d1-4ca7-bb6e-45032e701414 nodeName:}" failed. No retries permitted until 2026-09-14 02:53:10.388982747 +0000 UTC m=+53.175044638 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5zdn2" (UID: "a890518d-c1d1-4ca7-bb6e-45032e701414") : configmap "kube-rbac-proxy" not found
Sep 14 02:53:02.389265 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:02.389099    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls podName:a890518d-c1d1-4ca7-bb6e-45032e701414 nodeName:}" failed. No retries permitted until 2026-09-14 02:53:10.389086531 +0000 UTC m=+53.175148423 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5zdn2" (UID: "a890518d-c1d1-4ca7-bb6e-45032e701414") : secret "insights-runtime-extractor-tls" not found
Sep 14 02:53:02.833038 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.833006    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-796c6999d5-m4fs4"]
Sep 14 02:53:02.868124 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.868092    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-796c6999d5-m4fs4"]
Sep 14 02:53:02.868292 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.868223    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:02.871572 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.871553    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pll7v\""
Sep 14 02:53:02.871718 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.871684    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 14 02:53:02.872394 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.872369    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 14 02:53:02.872490 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.872434    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 14 02:53:02.879586 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.879569    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 14 02:53:02.995296 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.995252    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8hv4\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-kube-api-access-m8hv4\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:02.995445 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.995312    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c95fc0c6-b606-4784-a7b1-9a9e4be67357-ca-trust-extracted\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:02.995445 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.995378    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c95fc0c6-b606-4784-a7b1-9a9e4be67357-trusted-ca\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:02.995445 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.995407    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-bound-sa-token\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:02.995445 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.995433    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c95fc0c6-b606-4784-a7b1-9a9e4be67357-image-registry-private-configuration\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:02.995611 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.995521    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c95fc0c6-b606-4784-a7b1-9a9e4be67357-installation-pull-secrets\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:02.995611 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.995550    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-certificates\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:02.995611 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:02.995576    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.035313 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.035280    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4q7sw" event={"ID":"fb9239b3-0ce7-4964-b4f5-b1f7547582b3","Type":"ContainerStarted","Data":"51b4dbb6489748aa1159473bdfd9475726936e428e137f1a88c4e4c6b265ccc8"}
Sep 14 02:53:03.035496 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.035348    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4q7sw" event={"ID":"fb9239b3-0ce7-4964-b4f5-b1f7547582b3","Type":"ContainerStarted","Data":"8ff752b64cae42bf34ce1ace4c038d6e66f31f19e2461fad7e8b7dce53903f4b"}
Sep 14 02:53:03.052731 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.052678    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-4q7sw" podStartSLOduration=3.407602903 podStartE2EDuration="5.052665916s" totalImagesPullingTime="1.645063013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:03.052189698 +0000 UTC m=+45.838251602" watchObservedRunningTime="2026-09-14 02:53:03.052665916 +0000 UTC m=+45.838727861"
Sep 14 02:53:03.096733 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.096666    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c95fc0c6-b606-4784-a7b1-9a9e4be67357-installation-pull-secrets\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.096733 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.096694    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-certificates\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.096733 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.096716    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.096914 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:03.096801    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 02:53:03.096914 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:03.096815    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-796c6999d5-m4fs4: secret "image-registry-tls" not found
Sep 14 02:53:03.096914 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.096856    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8hv4\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-kube-api-access-m8hv4\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.096914 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:03.096866    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls podName:c95fc0c6-b606-4784-a7b1-9a9e4be67357 nodeName:}" failed. No retries permitted until 2026-09-14 02:53:03.596849108 +0000 UTC m=+46.382911013 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls") pod "image-registry-796c6999d5-m4fs4" (UID: "c95fc0c6-b606-4784-a7b1-9a9e4be67357") : secret "image-registry-tls" not found
Sep 14 02:53:03.097092 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.096918    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c95fc0c6-b606-4784-a7b1-9a9e4be67357-ca-trust-extracted\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.097092 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.096955    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c95fc0c6-b606-4784-a7b1-9a9e4be67357-trusted-ca\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.097092 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.096979    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-bound-sa-token\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.097092 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.097012    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c95fc0c6-b606-4784-a7b1-9a9e4be67357-image-registry-private-configuration\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.097388 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.097368    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c95fc0c6-b606-4784-a7b1-9a9e4be67357-ca-trust-extracted\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.097632 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.097593    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-certificates\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.097877 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.097857    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c95fc0c6-b606-4784-a7b1-9a9e4be67357-trusted-ca\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.099253 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.099228    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c95fc0c6-b606-4784-a7b1-9a9e4be67357-installation-pull-secrets\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.099393 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.099376    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c95fc0c6-b606-4784-a7b1-9a9e4be67357-image-registry-private-configuration\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.107889 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.107870    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-bound-sa-token\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.109435 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.109415    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8hv4\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-kube-api-access-m8hv4\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.600332 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:03.600301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:03.600683 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:03.600438    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 02:53:03.600683 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:03.600448    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-796c6999d5-m4fs4: secret "image-registry-tls" not found
Sep 14 02:53:03.600683 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:03.600507    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls podName:c95fc0c6-b606-4784-a7b1-9a9e4be67357 nodeName:}" failed. No retries permitted until 2026-09-14 02:53:04.600492537 +0000 UTC m=+47.386554428 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls") pod "image-registry-796c6999d5-m4fs4" (UID: "c95fc0c6-b606-4784-a7b1-9a9e4be67357") : secret "image-registry-tls" not found
Sep 14 02:53:04.270369 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.270332    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-r6t2n"]
Sep 14 02:53:04.300283 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.300257    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-r6t2n"]
Sep 14 02:53:04.300405 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.300347    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-r6t2n"
Sep 14 02:53:04.303168 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.303145    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-mmwls\""
Sep 14 02:53:04.303328 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.303245    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 14 02:53:04.303328 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.303278    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 14 02:53:04.304334 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.304314    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 14 02:53:04.304408 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.304322    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 14 02:53:04.405784 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.405757    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c8863421-adeb-4435-a94a-5ecea5a9d0ac-signing-cabundle\") pod \"service-ca-568869fdc5-r6t2n\" (UID: \"c8863421-adeb-4435-a94a-5ecea5a9d0ac\") " pod="openshift-service-ca/service-ca-568869fdc5-r6t2n"
Sep 14 02:53:04.405898 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.405807    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c8863421-adeb-4435-a94a-5ecea5a9d0ac-signing-key\") pod \"service-ca-568869fdc5-r6t2n\" (UID: \"c8863421-adeb-4435-a94a-5ecea5a9d0ac\") " pod="openshift-service-ca/service-ca-568869fdc5-r6t2n"
Sep 14 02:53:04.405898 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.405833    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88c9l\" (UniqueName: \"kubernetes.io/projected/c8863421-adeb-4435-a94a-5ecea5a9d0ac-kube-api-access-88c9l\") pod \"service-ca-568869fdc5-r6t2n\" (UID: \"c8863421-adeb-4435-a94a-5ecea5a9d0ac\") " pod="openshift-service-ca/service-ca-568869fdc5-r6t2n"
Sep 14 02:53:04.506553 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.506531    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c8863421-adeb-4435-a94a-5ecea5a9d0ac-signing-cabundle\") pod \"service-ca-568869fdc5-r6t2n\" (UID: \"c8863421-adeb-4435-a94a-5ecea5a9d0ac\") " pod="openshift-service-ca/service-ca-568869fdc5-r6t2n"
Sep 14 02:53:04.506662 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.506576    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c8863421-adeb-4435-a94a-5ecea5a9d0ac-signing-key\") pod \"service-ca-568869fdc5-r6t2n\" (UID: \"c8863421-adeb-4435-a94a-5ecea5a9d0ac\") " pod="openshift-service-ca/service-ca-568869fdc5-r6t2n"
Sep 14 02:53:04.506662 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.506598    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88c9l\" (UniqueName: \"kubernetes.io/projected/c8863421-adeb-4435-a94a-5ecea5a9d0ac-kube-api-access-88c9l\") pod \"service-ca-568869fdc5-r6t2n\" (UID: \"c8863421-adeb-4435-a94a-5ecea5a9d0ac\") " pod="openshift-service-ca/service-ca-568869fdc5-r6t2n"
Sep 14 02:53:04.507129 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.507105    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c8863421-adeb-4435-a94a-5ecea5a9d0ac-signing-cabundle\") pod \"service-ca-568869fdc5-r6t2n\" (UID: \"c8863421-adeb-4435-a94a-5ecea5a9d0ac\") " pod="openshift-service-ca/service-ca-568869fdc5-r6t2n"
Sep 14 02:53:04.509089 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.509060    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c8863421-adeb-4435-a94a-5ecea5a9d0ac-signing-key\") pod \"service-ca-568869fdc5-r6t2n\" (UID: \"c8863421-adeb-4435-a94a-5ecea5a9d0ac\") " pod="openshift-service-ca/service-ca-568869fdc5-r6t2n"
Sep 14 02:53:04.515259 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.515241    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88c9l\" (UniqueName: \"kubernetes.io/projected/c8863421-adeb-4435-a94a-5ecea5a9d0ac-kube-api-access-88c9l\") pod \"service-ca-568869fdc5-r6t2n\" (UID: \"c8863421-adeb-4435-a94a-5ecea5a9d0ac\") " pod="openshift-service-ca/service-ca-568869fdc5-r6t2n"
Sep 14 02:53:04.607965 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.607915    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:04.608248 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:04.608049    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 02:53:04.608248 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:04.608064    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-796c6999d5-m4fs4: secret "image-registry-tls" not found
Sep 14 02:53:04.608248 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:04.608122    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls podName:c95fc0c6-b606-4784-a7b1-9a9e4be67357 nodeName:}" failed. No retries permitted until 2026-09-14 02:53:06.608109127 +0000 UTC m=+49.394171018 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls") pod "image-registry-796c6999d5-m4fs4" (UID: "c95fc0c6-b606-4784-a7b1-9a9e4be67357") : secret "image-registry-tls" not found
Sep 14 02:53:04.608826 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.608810    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-r6t2n"
Sep 14 02:53:04.902870 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:04.902797    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-r6t2n"]
Sep 14 02:53:04.906207 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:53:04.906159    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8863421_adeb_4435_a94a_5ecea5a9d0ac.slice/crio-f13f468f6c4c3489435777d2ecbdb185645db57a9d6485730af909cf0cbf2da1 WatchSource:0}: Error finding container f13f468f6c4c3489435777d2ecbdb185645db57a9d6485730af909cf0cbf2da1: Status 404 returned error can't find the container with id f13f468f6c4c3489435777d2ecbdb185645db57a9d6485730af909cf0cbf2da1
Sep 14 02:53:05.042793 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:05.042759    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-r6t2n" event={"ID":"c8863421-adeb-4435-a94a-5ecea5a9d0ac","Type":"ContainerStarted","Data":"0f59728ca9b309d0b223e03b50c6ddea7626853d67c7c4d205e221132c745e8d"}
Sep 14 02:53:05.042940 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:05.042820    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-r6t2n" event={"ID":"c8863421-adeb-4435-a94a-5ecea5a9d0ac","Type":"ContainerStarted","Data":"f13f468f6c4c3489435777d2ecbdb185645db57a9d6485730af909cf0cbf2da1"}
Sep 14 02:53:05.063681 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:05.063639    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-r6t2n" podStartSLOduration=1.063623817 podStartE2EDuration="1.063623817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:53:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:05.062659117 +0000 UTC m=+47.848721051" watchObservedRunningTime="2026-09-14 02:53:05.063623817 +0000 UTC m=+47.849685727"
Sep 14 02:53:06.628055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:06.628014    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:06.628434 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:06.628146    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 02:53:06.628434 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:06.628165    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-796c6999d5-m4fs4: secret "image-registry-tls" not found
Sep 14 02:53:06.628434 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:06.628233    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls podName:c95fc0c6-b606-4784-a7b1-9a9e4be67357 nodeName:}" failed. No retries permitted until 2026-09-14 02:53:10.628219118 +0000 UTC m=+53.414281013 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls") pod "image-registry-796c6999d5-m4fs4" (UID: "c95fc0c6-b606-4784-a7b1-9a9e4be67357") : secret "image-registry-tls" not found
Sep 14 02:53:08.543158 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:08.543124    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:53:08.543542 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:08.543269    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 02:53:08.543542 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:08.543329    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls podName:56e1da36-b3ce-4cf4-b99c-dd9a7822c73c nodeName:}" failed. No retries permitted until 2026-09-14 02:53:24.543313007 +0000 UTC m=+67.329374898 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls") pod "dns-default-pq276" (UID: "56e1da36-b3ce-4cf4-b99c-dd9a7822c73c") : secret "dns-default-metrics-tls" not found
Sep 14 02:53:10.457486 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:10.457443    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:53:10.457993 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:10.457530    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:53:10.457993 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:10.457588    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 02:53:10.457993 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:10.457653    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls podName:a890518d-c1d1-4ca7-bb6e-45032e701414 nodeName:}" failed. No retries permitted until 2026-09-14 02:53:26.457638618 +0000 UTC m=+69.243700509 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5zdn2" (UID: "a890518d-c1d1-4ca7-bb6e-45032e701414") : secret "insights-runtime-extractor-tls" not found
Sep 14 02:53:10.458139 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:10.458119    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a890518d-c1d1-4ca7-bb6e-45032e701414-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:53:10.659152 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:10.659110    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:10.659326 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:10.659294    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 02:53:10.659326 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:10.659316    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-796c6999d5-m4fs4: secret "image-registry-tls" not found
Sep 14 02:53:10.659409 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:10.659381    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls podName:c95fc0c6-b606-4784-a7b1-9a9e4be67357 nodeName:}" failed. No retries permitted until 2026-09-14 02:53:18.659360146 +0000 UTC m=+61.445422041 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls") pod "image-registry-796c6999d5-m4fs4" (UID: "c95fc0c6-b606-4784-a7b1-9a9e4be67357") : secret "image-registry-tls" not found
Sep 14 02:53:17.973771 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:17.973739    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zvqjb"
Sep 14 02:53:18.716908 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:18.716869    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:18.720150 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:18.720124    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls\") pod \"image-registry-796c6999d5-m4fs4\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") " pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:18.780019 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:18.779998    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pll7v\""
Sep 14 02:53:18.787730 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:18.787713    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:19.088122 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:19.088087    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-796c6999d5-m4fs4"]
Sep 14 02:53:20.099429 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:20.099394    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-796c6999d5-m4fs4" event={"ID":"c95fc0c6-b606-4784-a7b1-9a9e4be67357","Type":"ContainerStarted","Data":"7e02bc2145228a6803a90a4d50b07d17c725f1dacbbae987d29c7d245aac29bc"}
Sep 14 02:53:20.099783 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:20.099442    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-796c6999d5-m4fs4" event={"ID":"c95fc0c6-b606-4784-a7b1-9a9e4be67357","Type":"ContainerStarted","Data":"14bb2d3ef2b9da0088e8f4a5352d9f8a42e79eba16f65a28068ff8a190e1f617"}
Sep 14 02:53:20.099783 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:20.099534    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:20.120427 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:20.120389    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-796c6999d5-m4fs4" podStartSLOduration=18.12037666 podStartE2EDuration="18.12037666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:53:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:20.119517128 +0000 UTC m=+62.905579043" watchObservedRunningTime="2026-09-14 02:53:20.12037666 +0000 UTC m=+62.906438573"
Sep 14 02:53:23.450429 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:23.450392    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret\") pod \"global-pull-secret-syncer-h5wrd\" (UID: \"2407936d-8e27-4f88-93a8-9dca1a2b80b9\") " pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:53:23.453652 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:23.453633    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 02:53:23.463363 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:23.463346    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2407936d-8e27-4f88-93a8-9dca1a2b80b9-original-pull-secret\") pod \"global-pull-secret-syncer-h5wrd\" (UID: \"2407936d-8e27-4f88-93a8-9dca1a2b80b9\") " pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:53:23.550894 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:23.550853    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs\") pod \"network-metrics-daemon-ct7gq\" (UID: \"35e90f11-d246-46e8-aa8b-ca9448f164e3\") " pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:53:23.551026 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:23.550940    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4wtg\" (UniqueName: \"kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg\") pod \"network-check-target-484dv\" (UID: \"fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4\") " pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:53:23.553536 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:23.553506    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4wtg\" (UniqueName: \"kubernetes.io/projected/fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4-kube-api-access-m4wtg\") pod \"network-check-target-484dv\" (UID: \"fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4\") " pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:53:23.553629 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:23.553610    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 02:53:23.563799 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:23.563770    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35e90f11-d246-46e8-aa8b-ca9448f164e3-metrics-certs\") pod \"network-metrics-daemon-ct7gq\" (UID: \"35e90f11-d246-46e8-aa8b-ca9448f164e3\") " pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:53:23.647213 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:23.647159    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h5wrd"
Sep 14 02:53:23.657863 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:23.657843    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lx47r\""
Sep 14 02:53:23.663212 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:23.663194    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hn74x\""
Sep 14 02:53:23.666198 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:23.666160    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ct7gq"
Sep 14 02:53:23.672097 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:23.672077    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:53:23.922055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:23.922025    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h5wrd"]
Sep 14 02:53:23.925031 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:53:23.925008    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2407936d_8e27_4f88_93a8_9dca1a2b80b9.slice/crio-531588b7fb05226787bb28ee6da42447aab86feb49ee6474e900c5d01d4cb2d6 WatchSource:0}: Error finding container 531588b7fb05226787bb28ee6da42447aab86feb49ee6474e900c5d01d4cb2d6: Status 404 returned error can't find the container with id 531588b7fb05226787bb28ee6da42447aab86feb49ee6474e900c5d01d4cb2d6
Sep 14 02:53:23.946866 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:23.946848    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-484dv"]
Sep 14 02:53:23.948970 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:53:23.948949    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc5ea33c_be3e_4a6e_a88f_1bc2d533e3b4.slice/crio-5e8544f688e94fe88147d4a8298e9f089d5115bc840a8dce3299a898f51dfe4f WatchSource:0}: Error finding container 5e8544f688e94fe88147d4a8298e9f089d5115bc840a8dce3299a898f51dfe4f: Status 404 returned error can't find the container with id 5e8544f688e94fe88147d4a8298e9f089d5115bc840a8dce3299a898f51dfe4f
Sep 14 02:53:23.952991 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:23.952901    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ct7gq"]
Sep 14 02:53:23.956279 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:53:23.956260    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e90f11_d246_46e8_aa8b_ca9448f164e3.slice/crio-222582b064cede5a0d3d8d6b3482e4d509183d49576475f82773deb145b3b53d WatchSource:0}: Error finding container 222582b064cede5a0d3d8d6b3482e4d509183d49576475f82773deb145b3b53d: Status 404 returned error can't find the container with id 222582b064cede5a0d3d8d6b3482e4d509183d49576475f82773deb145b3b53d
Sep 14 02:53:24.113601 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:24.113571    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ct7gq" event={"ID":"35e90f11-d246-46e8-aa8b-ca9448f164e3","Type":"ContainerStarted","Data":"222582b064cede5a0d3d8d6b3482e4d509183d49576475f82773deb145b3b53d"}
Sep 14 02:53:24.115119 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:24.115090    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-484dv" event={"ID":"fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4","Type":"ContainerStarted","Data":"09f5275ba48a65b17d3d9500e321268cf7f7c6d49900733b20def4cd3132550f"}
Sep 14 02:53:24.115243 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:24.115129    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-484dv" event={"ID":"fc5ea33c-be3e-4a6e-a88f-1bc2d533e3b4","Type":"ContainerStarted","Data":"5e8544f688e94fe88147d4a8298e9f089d5115bc840a8dce3299a898f51dfe4f"}
Sep 14 02:53:24.115243 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:24.115214    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:53:24.116231 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:24.116211    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h5wrd" event={"ID":"2407936d-8e27-4f88-93a8-9dca1a2b80b9","Type":"ContainerStarted","Data":"531588b7fb05226787bb28ee6da42447aab86feb49ee6474e900c5d01d4cb2d6"}
Sep 14 02:53:24.559225 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:24.559119    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:53:24.562325 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:24.562276    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56e1da36-b3ce-4cf4-b99c-dd9a7822c73c-metrics-tls\") pod \"dns-default-pq276\" (UID: \"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c\") " pod="openshift-dns/dns-default-pq276"
Sep 14 02:53:24.753647 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:24.753618    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gj97m\""
Sep 14 02:53:24.761158 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:24.760942    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pq276"
Sep 14 02:53:25.122497 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:25.122462    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ct7gq" event={"ID":"35e90f11-d246-46e8-aa8b-ca9448f164e3","Type":"ContainerStarted","Data":"5d2618425d3c1669b1bceedc2974049537fb890fda8933705228e43707035dad"}
Sep 14 02:53:25.181275 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:25.181146    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-484dv" podStartSLOduration=67.181127781 podStartE2EDuration="1m7.181127781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:24.132594102 +0000 UTC m=+66.918656015" watchObservedRunningTime="2026-09-14 02:53:25.181127781 +0000 UTC m=+67.967189694"
Sep 14 02:53:25.182053 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:25.182028    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pq276"]
Sep 14 02:53:25.186186 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:53:25.186144    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56e1da36_b3ce_4cf4_b99c_dd9a7822c73c.slice/crio-28dab4375f53ab038f79f2540b3763e716812ad9f5e7219491526e11fed1d6a9 WatchSource:0}: Error finding container 28dab4375f53ab038f79f2540b3763e716812ad9f5e7219491526e11fed1d6a9: Status 404 returned error can't find the container with id 28dab4375f53ab038f79f2540b3763e716812ad9f5e7219491526e11fed1d6a9
Sep 14 02:53:26.127938 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:26.127904    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pq276" event={"ID":"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c","Type":"ContainerStarted","Data":"28dab4375f53ab038f79f2540b3763e716812ad9f5e7219491526e11fed1d6a9"}
Sep 14 02:53:26.130214 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:26.130164    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ct7gq" event={"ID":"35e90f11-d246-46e8-aa8b-ca9448f164e3","Type":"ContainerStarted","Data":"cf1d5053bdc93e5f39f055e90d890965a7b562175c808df29f77980d5ab0219a"}
Sep 14 02:53:26.149074 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:26.149008    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ct7gq" podStartSLOduration=67.173884974 podStartE2EDuration="1m8.14898731s" totalImagesPullingTime="975.102336ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:26.147099751 +0000 UTC m=+68.933161674" watchObservedRunningTime="2026-09-14 02:53:26.14898731 +0000 UTC m=+68.935049222"
Sep 14 02:53:26.476901 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:26.476814    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:53:26.479624 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:26.479602    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a890518d-c1d1-4ca7-bb6e-45032e701414-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5zdn2\" (UID: \"a890518d-c1d1-4ca7-bb6e-45032e701414\") " pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:53:26.684525 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:26.684490    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ztclm\""
Sep 14 02:53:26.692785 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:26.692748    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5zdn2"
Sep 14 02:53:28.365050 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:53:28.365018    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda890518d_c1d1_4ca7_bb6e_45032e701414.slice/crio-f4b08ac16bd986fb159dc1dcaba947b45d43cba6c90383ff93ac41a3cf349832 WatchSource:0}: Error finding container f4b08ac16bd986fb159dc1dcaba947b45d43cba6c90383ff93ac41a3cf349832: Status 404 returned error can't find the container with id f4b08ac16bd986fb159dc1dcaba947b45d43cba6c90383ff93ac41a3cf349832
Sep 14 02:53:28.366019 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:28.365995    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5zdn2"]
Sep 14 02:53:29.145572 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.145533    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h5wrd" event={"ID":"2407936d-8e27-4f88-93a8-9dca1a2b80b9","Type":"ContainerStarted","Data":"9e14ce6b320eb3488b133a0c145706be153ec4c9c89601e12c119118f3bc3491"}
Sep 14 02:53:29.147344 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.147318    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pq276" event={"ID":"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c","Type":"ContainerStarted","Data":"48dd5d954be30d64ed677944d7b2f0178d6cd363b7769e11ad60e317d74daef9"}
Sep 14 02:53:29.147464 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.147364    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pq276" event={"ID":"56e1da36-b3ce-4cf4-b99c-dd9a7822c73c","Type":"ContainerStarted","Data":"3f7183ef4968dac4afc249884be626e270b7edd9e621d73bab80de561e0f6044"}
Sep 14 02:53:29.147464 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.147394    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pq276"
Sep 14 02:53:29.148851 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.148832    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5zdn2" event={"ID":"a890518d-c1d1-4ca7-bb6e-45032e701414","Type":"ContainerStarted","Data":"0fbfa5476817907440945549c38fd8a8a820fe0b78605008e466fe55c91c8411"}
Sep 14 02:53:29.148928 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.148877    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5zdn2" event={"ID":"a890518d-c1d1-4ca7-bb6e-45032e701414","Type":"ContainerStarted","Data":"f4b08ac16bd986fb159dc1dcaba947b45d43cba6c90383ff93ac41a3cf349832"}
Sep 14 02:53:29.161490 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.161454    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h5wrd" podStartSLOduration=67.004753321 podStartE2EDuration="1m11.161440247s" totalImagesPullingTime="4.156686926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:52:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:29.160409265 +0000 UTC m=+71.946471178" watchObservedRunningTime="2026-09-14 02:53:29.161440247 +0000 UTC m=+71.947502159"
Sep 14 02:53:29.179718 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.179682    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pq276" podStartSLOduration=34.280032387 podStartE2EDuration="37.179668536s" totalImagesPullingTime="2.899636149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:52:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:29.178661208 +0000 UTC m=+71.964723135" watchObservedRunningTime="2026-09-14 02:53:29.179668536 +0000 UTC m=+71.965730448"
Sep 14 02:53:29.698114 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.698048    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-2pxbm"]
Sep 14 02:53:29.718058 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.718039    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2pxbm"
Sep 14 02:53:29.722895 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.722872    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 14 02:53:29.723090 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.723070    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 14 02:53:29.723225 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.723156    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hq8kj\""
Sep 14 02:53:29.725850 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.725829    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-2pxbm"]
Sep 14 02:53:29.743884 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.743863    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"]
Sep 14 02:53:29.761439 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.761419    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.764325 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.764304    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 14 02:53:29.764464 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.764352    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 14 02:53:29.764464 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.764375    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 14 02:53:29.764574 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.764492    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 14 02:53:29.764730 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.764713    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 14 02:53:29.764813 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.764756    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-796c6999d5-m4fs4"]
Sep 14 02:53:29.764867 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.764817    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 14 02:53:29.764928 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.764910    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 14 02:53:29.769868 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.769834    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"]
Sep 14 02:53:29.797556 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.797530    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mc26m"]
Sep 14 02:53:29.799759 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.799738    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b7851e60-80bc-4bdc-871c-acd2d69548b5-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-2pxbm\" (UID: \"b7851e60-80bc-4bdc-871c-acd2d69548b5\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2pxbm"
Sep 14 02:53:29.799845 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.799769    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-hub\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.799845 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.799788    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-additional-service-ca\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.799845 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.799813    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.799941 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.799869    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.799941 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.799898    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7851e60-80bc-4bdc-871c-acd2d69548b5-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2pxbm\" (UID: \"b7851e60-80bc-4bdc-871c-acd2d69548b5\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2pxbm"
Sep 14 02:53:29.799941 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.799920    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-ca\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.800032 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.799961    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv8w4\" (UniqueName: \"kubernetes.io/projected/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-kube-api-access-kv8w4\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.811440 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.811415    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mc26m"
Sep 14 02:53:29.814443 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.814219    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 14 02:53:29.815479 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.815223    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mc26m"]
Sep 14 02:53:29.815578 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.815564    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-wh8pd\""
Sep 14 02:53:29.815988 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.815970    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-57bff6854f-2wdw2"]
Sep 14 02:53:29.838197 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.838154    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:29.845523 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.845501    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57bff6854f-2wdw2"]
Sep 14 02:53:29.900507 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.900467    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7851e60-80bc-4bdc-871c-acd2d69548b5-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2pxbm\" (UID: \"b7851e60-80bc-4bdc-871c-acd2d69548b5\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2pxbm"
Sep 14 02:53:29.900729 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.900517    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/40fdddc8-0da1-4037-8c59-8b616f5ee781-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-mc26m\" (UID: \"40fdddc8-0da1-4037-8c59-8b616f5ee781\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mc26m"
Sep 14 02:53:29.900729 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.900551    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-registry-certificates\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:29.900729 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.900578    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-ca\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.900729 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.900615    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-image-registry-private-configuration\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:29.900729 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.900639    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-bound-sa-token\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:29.900729 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.900663    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-ca-trust-extracted\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:29.900978 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.900743    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-installation-pull-secrets\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:29.900978 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.900781    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js724\" (UniqueName: \"kubernetes.io/projected/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-kube-api-access-js724\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:29.900978 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.900836    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv8w4\" (UniqueName: \"kubernetes.io/projected/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-kube-api-access-kv8w4\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.900978 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.900865    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-trusted-ca\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:29.900978 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.900948    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-registry-tls\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:29.901254 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.901000    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b7851e60-80bc-4bdc-871c-acd2d69548b5-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-2pxbm\" (UID: \"b7851e60-80bc-4bdc-871c-acd2d69548b5\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2pxbm"
Sep 14 02:53:29.901254 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.901041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-hub\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.901254 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.901067    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-additional-service-ca\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.901254 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.901091    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.901254 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.901117    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.902053 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.901716    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b7851e60-80bc-4bdc-871c-acd2d69548b5-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-2pxbm\" (UID: \"b7851e60-80bc-4bdc-871c-acd2d69548b5\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2pxbm"
Sep 14 02:53:29.902053 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.901740    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-additional-service-ca\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.903690 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.903669    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7851e60-80bc-4bdc-871c-acd2d69548b5-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2pxbm\" (UID: \"b7851e60-80bc-4bdc-871c-acd2d69548b5\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2pxbm"
Sep 14 02:53:29.904489 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.904441    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-hub\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.904489 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.904463    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.904649 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.904618    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-ca\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.904729 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.904712    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:29.913887 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:29.913859    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv8w4\" (UniqueName: \"kubernetes.io/projected/b54d39b6-abce-4e8d-bc04-f3671e2c1f41-kube-api-access-kv8w4\") pod \"cluster-proxy-proxy-agent-74585885f7-dlfn7\" (UID: \"b54d39b6-abce-4e8d-bc04-f3671e2c1f41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:30.002427 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.002390    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-trusted-ca\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.002598 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.002440    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-registry-tls\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.002598 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.002500    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/40fdddc8-0da1-4037-8c59-8b616f5ee781-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-mc26m\" (UID: \"40fdddc8-0da1-4037-8c59-8b616f5ee781\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mc26m"
Sep 14 02:53:30.002598 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.002532    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-registry-certificates\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.002598 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.002568    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-image-registry-private-configuration\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.002800 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.002611    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-bound-sa-token\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.002800 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.002638    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-ca-trust-extracted\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.002800 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.002672    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-installation-pull-secrets\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.002800 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.002699    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js724\" (UniqueName: \"kubernetes.io/projected/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-kube-api-access-js724\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.003158 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.003124    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-ca-trust-extracted\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.003402 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.003380    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-registry-certificates\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.003498 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.003391    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-trusted-ca\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.005566 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.005541    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/40fdddc8-0da1-4037-8c59-8b616f5ee781-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-mc26m\" (UID: \"40fdddc8-0da1-4037-8c59-8b616f5ee781\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mc26m"
Sep 14 02:53:30.005949 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.005910    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-image-registry-private-configuration\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.006409 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.006384    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-registry-tls\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.006499 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.006403    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-installation-pull-secrets\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.012801 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.012777    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js724\" (UniqueName: \"kubernetes.io/projected/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-kube-api-access-js724\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.013037 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.013017    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f0eb029-15e9-43c1-82ca-19ac255e4c5b-bound-sa-token\") pod \"image-registry-57bff6854f-2wdw2\" (UID: \"7f0eb029-15e9-43c1-82ca-19ac255e4c5b\") " pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.027934 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.027911    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2pxbm"
Sep 14 02:53:30.081955 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.081920    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"
Sep 14 02:53:30.122811 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.122746    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mc26m"
Sep 14 02:53:30.154148 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.152703    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:30.163744 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.163627    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5zdn2" event={"ID":"a890518d-c1d1-4ca7-bb6e-45032e701414","Type":"ContainerStarted","Data":"d803c26af11e085ad75d2b590b9680833bbb1e52bf889ac6bc8b570a37b1be31"}
Sep 14 02:53:30.335324 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.335258    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-2pxbm"]
Sep 14 02:53:30.338131 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:53:30.338100    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7851e60_80bc_4bdc_871c_acd2d69548b5.slice/crio-23e60fe7830a3332f92bb7ae153af91369ad1e9abccc1899664062942ccb42eb WatchSource:0}: Error finding container 23e60fe7830a3332f92bb7ae153af91369ad1e9abccc1899664062942ccb42eb: Status 404 returned error can't find the container with id 23e60fe7830a3332f92bb7ae153af91369ad1e9abccc1899664062942ccb42eb
Sep 14 02:53:30.392834 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.392811    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7"]
Sep 14 02:53:30.395880 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:53:30.395855    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb54d39b6_abce_4e8d_bc04_f3671e2c1f41.slice/crio-b28854939048d89820dd3e4393a1afcebfd5a290a3dbfe01ac080024a8780e29 WatchSource:0}: Error finding container b28854939048d89820dd3e4393a1afcebfd5a290a3dbfe01ac080024a8780e29: Status 404 returned error can't find the container with id b28854939048d89820dd3e4393a1afcebfd5a290a3dbfe01ac080024a8780e29
Sep 14 02:53:30.422479 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.422409    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mc26m"]
Sep 14 02:53:30.463258 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:30.463222    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57bff6854f-2wdw2"]
Sep 14 02:53:30.881215 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:53:30.881159    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40fdddc8_0da1_4037_8c59_8b616f5ee781.slice/crio-901a250c92bb98cdb42cb277ad8d98f66bc4a787849a622a48d00fc771662a11 WatchSource:0}: Error finding container 901a250c92bb98cdb42cb277ad8d98f66bc4a787849a622a48d00fc771662a11: Status 404 returned error can't find the container with id 901a250c92bb98cdb42cb277ad8d98f66bc4a787849a622a48d00fc771662a11
Sep 14 02:53:30.881788 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:53:30.881766    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f0eb029_15e9_43c1_82ca_19ac255e4c5b.slice/crio-b3401a777eedb5fc5e58ab4deedc7faa1ec44453efe2aac55014e97651ba4031 WatchSource:0}: Error finding container b3401a777eedb5fc5e58ab4deedc7faa1ec44453efe2aac55014e97651ba4031: Status 404 returned error can't find the container with id b3401a777eedb5fc5e58ab4deedc7faa1ec44453efe2aac55014e97651ba4031
Sep 14 02:53:31.169386 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:31.169305    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57bff6854f-2wdw2" event={"ID":"7f0eb029-15e9-43c1-82ca-19ac255e4c5b","Type":"ContainerStarted","Data":"f73001260ea84ba266b1b35a29df6d6f5af8ce138ae7840e3bedb39b32f34d3c"}
Sep 14 02:53:31.169386 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:31.169366    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57bff6854f-2wdw2" event={"ID":"7f0eb029-15e9-43c1-82ca-19ac255e4c5b","Type":"ContainerStarted","Data":"b3401a777eedb5fc5e58ab4deedc7faa1ec44453efe2aac55014e97651ba4031"}
Sep 14 02:53:31.169615 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:31.169392    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:31.170722 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:31.170699    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7" event={"ID":"b54d39b6-abce-4e8d-bc04-f3671e2c1f41","Type":"ContainerStarted","Data":"b28854939048d89820dd3e4393a1afcebfd5a290a3dbfe01ac080024a8780e29"}
Sep 14 02:53:31.171779 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:31.171754    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2pxbm" event={"ID":"b7851e60-80bc-4bdc-871c-acd2d69548b5","Type":"ContainerStarted","Data":"23e60fe7830a3332f92bb7ae153af91369ad1e9abccc1899664062942ccb42eb"}
Sep 14 02:53:31.173034 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:31.173010    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mc26m" event={"ID":"40fdddc8-0da1-4037-8c59-8b616f5ee781","Type":"ContainerStarted","Data":"901a250c92bb98cdb42cb277ad8d98f66bc4a787849a622a48d00fc771662a11"}
Sep 14 02:53:31.192898 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:31.192845    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-57bff6854f-2wdw2" podStartSLOduration=2.192830572 podStartE2EDuration="2.192830572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:53:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:31.190675162 +0000 UTC m=+73.976737067" watchObservedRunningTime="2026-09-14 02:53:31.192830572 +0000 UTC m=+73.978892485"
Sep 14 02:53:32.180575 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:32.179802    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5zdn2" event={"ID":"a890518d-c1d1-4ca7-bb6e-45032e701414","Type":"ContainerStarted","Data":"01cf93ade6efdedd30e01f88fc7283ced50b7031e91dc84922ef79be47d40ed9"}
Sep 14 02:53:32.230580 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:32.230534    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5zdn2" podStartSLOduration=35.53411046 podStartE2EDuration="38.230519334s" totalImagesPullingTime="2.696408874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 02:52:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:32.22990598 +0000 UTC m=+75.015967894" watchObservedRunningTime="2026-09-14 02:53:32.230519334 +0000 UTC m=+75.016581247"
Sep 14 02:53:35.194060 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:35.194023    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7" event={"ID":"b54d39b6-abce-4e8d-bc04-f3671e2c1f41","Type":"ContainerStarted","Data":"a5285863733c3335c560c68261ca98d87acb7873048f4c9417e3f9809c30ce73"}
Sep 14 02:53:35.194527 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:35.194068    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7" event={"ID":"b54d39b6-abce-4e8d-bc04-f3671e2c1f41","Type":"ContainerStarted","Data":"b6333ffeb29e08d0ab36fd6b73273d399114a2f5420e6e571f51f388f24a387a"}
Sep 14 02:53:35.194527 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:35.194084    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7" event={"ID":"b54d39b6-abce-4e8d-bc04-f3671e2c1f41","Type":"ContainerStarted","Data":"b97040ecf32d63021d14de30310328cefe8fa287404b3f160db56baddfe0f3ea"}
Sep 14 02:53:35.195575 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:35.195553    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2pxbm" event={"ID":"b7851e60-80bc-4bdc-871c-acd2d69548b5","Type":"ContainerStarted","Data":"89325cb4e132e3a3997e138ed71fe3f9c260de4b4dce4287115041f73cc31972"}
Sep 14 02:53:35.196955 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:35.196935    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mc26m" event={"ID":"40fdddc8-0da1-4037-8c59-8b616f5ee781","Type":"ContainerStarted","Data":"f6e11b308d50b21837f21914dd95db8564dd6f8e33a4b4de2f0292091e308f92"}
Sep 14 02:53:35.197192 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:35.197156    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mc26m"
Sep 14 02:53:35.201618 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:35.201600    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mc26m"
Sep 14 02:53:35.213187 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:35.213136    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74585885f7-dlfn7" podStartSLOduration=2.023316002 podStartE2EDuration="6.213104509s" totalImagesPullingTime="4.189788507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:53:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:35.211977636 +0000 UTC m=+77.998039547" watchObservedRunningTime="2026-09-14 02:53:35.213104509 +0000 UTC m=+77.999166413"
Sep 14 02:53:35.243023 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:35.242989    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2pxbm" podStartSLOduration=2.035542582 podStartE2EDuration="6.242977013s" totalImagesPullingTime="4.207434431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:53:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:35.22669144 +0000 UTC m=+78.012753353" watchObservedRunningTime="2026-09-14 02:53:35.242977013 +0000 UTC m=+78.029038961"
Sep 14 02:53:35.243155 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:35.243092    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-mc26m" podStartSLOduration=2.577355731 podStartE2EDuration="6.243088366s" totalImagesPullingTime="3.665732635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:53:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:35.242324503 +0000 UTC m=+78.028386418" watchObservedRunningTime="2026-09-14 02:53:35.243088366 +0000 UTC m=+78.029150280"
Sep 14 02:53:39.166128 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:39.165998    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pq276"
Sep 14 02:53:39.770594 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:39.770566    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:41.496280 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.496247    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-88lxg"]
Sep 14 02:53:41.501487 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.501466    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.505866 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.505841    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 14 02:53:41.507002 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.506984    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 14 02:53:41.507111 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.507033    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 14 02:53:41.507208 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.506985    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 14 02:53:41.507278 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.507241    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 14 02:53:41.507358 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.507342    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xb6b2\""
Sep 14 02:53:41.508167 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.508135    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 14 02:53:41.592703 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.592678    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-sys\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.592851 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.592711    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-textfile\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.592851 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.592759    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfwgw\" (UniqueName: \"kubernetes.io/projected/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-kube-api-access-dfwgw\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.592851 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.592829    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-tls\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.592972 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.592926    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.593012 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.592978    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-wtmp\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.593054 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.593009    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-root\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.593054 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.593035    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-accelerators-collector-config\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.593154 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.593106    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-metrics-client-ca\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.693969 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.693940    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.694127 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.693990    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-wtmp\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.694127 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.694019    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-root\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.694127 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.694045    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-accelerators-collector-config\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.694127 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.694076    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-metrics-client-ca\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.694127 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.694110    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-sys\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.694420 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.694134    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-textfile\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.694420 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.694155    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-wtmp\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.694420 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.694197    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfwgw\" (UniqueName: \"kubernetes.io/projected/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-kube-api-access-dfwgw\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.694420 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.694225    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-tls\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.694420 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.694273    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-root\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.694420 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:41.694343    2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 14 02:53:41.694420 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:41.694399    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-tls podName:4e5107dd-669d-4e75-8a27-63b58d6fd6b5 nodeName:}" failed. No retries permitted until 2026-09-14 02:53:42.19437956 +0000 UTC m=+84.980441458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-tls") pod "node-exporter-88lxg" (UID: "4e5107dd-669d-4e75-8a27-63b58d6fd6b5") : secret "node-exporter-tls" not found
Sep 14 02:53:41.694798 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.694539    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-textfile\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.694798 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.694597    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-sys\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.694798 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.694761    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-metrics-client-ca\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.695073 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.695051    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-accelerators-collector-config\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.696915 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.696891    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:41.704198 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:41.704157    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfwgw\" (UniqueName: \"kubernetes.io/projected/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-kube-api-access-dfwgw\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:42.197502 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:42.197469    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-tls\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:42.199959 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:42.199931    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4e5107dd-669d-4e75-8a27-63b58d6fd6b5-node-exporter-tls\") pod \"node-exporter-88lxg\" (UID: \"4e5107dd-669d-4e75-8a27-63b58d6fd6b5\") " pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:42.413423 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:42.413391    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-88lxg"
Sep 14 02:53:42.426100 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:53:42.426072    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e5107dd_669d_4e75_8a27_63b58d6fd6b5.slice/crio-16c6237e84922ec29721ccd23e3f825cc221944eb7880efaf1a089422a9bd483 WatchSource:0}: Error finding container 16c6237e84922ec29721ccd23e3f825cc221944eb7880efaf1a089422a9bd483: Status 404 returned error can't find the container with id 16c6237e84922ec29721ccd23e3f825cc221944eb7880efaf1a089422a9bd483
Sep 14 02:53:43.229011 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:43.228971    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88lxg" event={"ID":"4e5107dd-669d-4e75-8a27-63b58d6fd6b5","Type":"ContainerStarted","Data":"16c6237e84922ec29721ccd23e3f825cc221944eb7880efaf1a089422a9bd483"}
Sep 14 02:53:44.234755 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:44.234721    2579 generic.go:356] "Generic (PLEG): container finished" podID="4e5107dd-669d-4e75-8a27-63b58d6fd6b5" containerID="902f43fdd90c5af81541029badd3fdd4afc8b7f36c8133ccec9a84e050909255" exitCode=0
Sep 14 02:53:44.235155 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:44.234790    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88lxg" event={"ID":"4e5107dd-669d-4e75-8a27-63b58d6fd6b5","Type":"ContainerDied","Data":"902f43fdd90c5af81541029badd3fdd4afc8b7f36c8133ccec9a84e050909255"}
Sep 14 02:53:45.239869 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:45.239833    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88lxg" event={"ID":"4e5107dd-669d-4e75-8a27-63b58d6fd6b5","Type":"ContainerStarted","Data":"318fa0e22f16220af66172038005ddf28a2d045ef62f481b0fed41fafa54e4eb"}
Sep 14 02:53:45.240253 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:45.239914    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88lxg" event={"ID":"4e5107dd-669d-4e75-8a27-63b58d6fd6b5","Type":"ContainerStarted","Data":"985db8a59309120c2f9130899f733de85eeb6fd569af6ca3af12d24c94ee2663"}
Sep 14 02:53:45.261486 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:45.261446    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-88lxg" podStartSLOduration=3.281895063 podStartE2EDuration="4.261431148s" totalImagesPullingTime="912.14198ms" totalInitContainerRuntime="67.394105ms" isStatefulPod=true podCreationTimestamp="2026-09-14 02:53:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:53:45.260366841 +0000 UTC m=+88.046428754" watchObservedRunningTime="2026-09-14 02:53:45.261431148 +0000 UTC m=+88.047493062"
Sep 14 02:53:52.184501 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:52.184468    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-57bff6854f-2wdw2"
Sep 14 02:53:54.785007 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:54.784967    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-796c6999d5-m4fs4" podUID="c95fc0c6-b606-4784-a7b1-9a9e4be67357" containerName="registry" containerID="cri-o://7e02bc2145228a6803a90a4d50b07d17c725f1dacbbae987d29c7d245aac29bc" gracePeriod=30
Sep 14 02:53:55.022160 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.022133    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:55.093644 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.093581    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-certificates\") pod \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") "
Sep 14 02:53:55.093644 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.093631    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c95fc0c6-b606-4784-a7b1-9a9e4be67357-ca-trust-extracted\") pod \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") "
Sep 14 02:53:55.093801 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.093671    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c95fc0c6-b606-4784-a7b1-9a9e4be67357-image-registry-private-configuration\") pod \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") "
Sep 14 02:53:55.093801 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.093701    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8hv4\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-kube-api-access-m8hv4\") pod \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") "
Sep 14 02:53:55.093801 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.093735    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-bound-sa-token\") pod \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") "
Sep 14 02:53:55.093939 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.093857    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls\") pod \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") "
Sep 14 02:53:55.093939 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.093924    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c95fc0c6-b606-4784-a7b1-9a9e4be67357-trusted-ca\") pod \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") "
Sep 14 02:53:55.094036 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.093955    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c95fc0c6-b606-4784-a7b1-9a9e4be67357-installation-pull-secrets\") pod \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\" (UID: \"c95fc0c6-b606-4784-a7b1-9a9e4be67357\") "
Sep 14 02:53:55.094036 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.094010    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c95fc0c6-b606-4784-a7b1-9a9e4be67357" (UID: "c95fc0c6-b606-4784-a7b1-9a9e4be67357"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:53:55.094258 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.094230    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-certificates\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 02:53:55.094934 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.094898    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c95fc0c6-b606-4784-a7b1-9a9e4be67357-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c95fc0c6-b606-4784-a7b1-9a9e4be67357" (UID: "c95fc0c6-b606-4784-a7b1-9a9e4be67357"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:53:55.096464 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.096432    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c95fc0c6-b606-4784-a7b1-9a9e4be67357" (UID: "c95fc0c6-b606-4784-a7b1-9a9e4be67357"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:53:55.096579 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.096539    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c95fc0c6-b606-4784-a7b1-9a9e4be67357-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c95fc0c6-b606-4784-a7b1-9a9e4be67357" (UID: "c95fc0c6-b606-4784-a7b1-9a9e4be67357"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:53:55.096648 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.096580    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c95fc0c6-b606-4784-a7b1-9a9e4be67357" (UID: "c95fc0c6-b606-4784-a7b1-9a9e4be67357"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:53:55.096707 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.096693    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c95fc0c6-b606-4784-a7b1-9a9e4be67357-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c95fc0c6-b606-4784-a7b1-9a9e4be67357" (UID: "c95fc0c6-b606-4784-a7b1-9a9e4be67357"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:53:55.096764 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.096737    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-kube-api-access-m8hv4" (OuterVolumeSpecName: "kube-api-access-m8hv4") pod "c95fc0c6-b606-4784-a7b1-9a9e4be67357" (UID: "c95fc0c6-b606-4784-a7b1-9a9e4be67357"). InnerVolumeSpecName "kube-api-access-m8hv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:53:55.103938 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.103912    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c95fc0c6-b606-4784-a7b1-9a9e4be67357-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c95fc0c6-b606-4784-a7b1-9a9e4be67357" (UID: "c95fc0c6-b606-4784-a7b1-9a9e4be67357"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:53:55.124595 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.124578    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-484dv"
Sep 14 02:53:55.195340 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.195318    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-registry-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 02:53:55.195340 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.195340    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c95fc0c6-b606-4784-a7b1-9a9e4be67357-trusted-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 02:53:55.195486 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.195353    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c95fc0c6-b606-4784-a7b1-9a9e4be67357-installation-pull-secrets\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 02:53:55.195486 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.195361    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c95fc0c6-b606-4784-a7b1-9a9e4be67357-ca-trust-extracted\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 02:53:55.195486 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.195370    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c95fc0c6-b606-4784-a7b1-9a9e4be67357-image-registry-private-configuration\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 02:53:55.195486 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.195379    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8hv4\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-kube-api-access-m8hv4\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 02:53:55.195486 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.195387    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c95fc0c6-b606-4784-a7b1-9a9e4be67357-bound-sa-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 02:53:55.279335 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.279305    2579 generic.go:356] "Generic (PLEG): container finished" podID="c95fc0c6-b606-4784-a7b1-9a9e4be67357" containerID="7e02bc2145228a6803a90a4d50b07d17c725f1dacbbae987d29c7d245aac29bc" exitCode=0
Sep 14 02:53:55.279519 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.279351    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-796c6999d5-m4fs4" event={"ID":"c95fc0c6-b606-4784-a7b1-9a9e4be67357","Type":"ContainerDied","Data":"7e02bc2145228a6803a90a4d50b07d17c725f1dacbbae987d29c7d245aac29bc"}
Sep 14 02:53:55.279519 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.279402    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-796c6999d5-m4fs4" event={"ID":"c95fc0c6-b606-4784-a7b1-9a9e4be67357","Type":"ContainerDied","Data":"14bb2d3ef2b9da0088e8f4a5352d9f8a42e79eba16f65a28068ff8a190e1f617"}
Sep 14 02:53:55.279519 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.279407    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-796c6999d5-m4fs4"
Sep 14 02:53:55.279519 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.279433    2579 scope.go:117] "RemoveContainer" containerID="7e02bc2145228a6803a90a4d50b07d17c725f1dacbbae987d29c7d245aac29bc"
Sep 14 02:53:55.288597 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.288581    2579 scope.go:117] "RemoveContainer" containerID="7e02bc2145228a6803a90a4d50b07d17c725f1dacbbae987d29c7d245aac29bc"
Sep 14 02:53:55.288868 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:53:55.288850    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e02bc2145228a6803a90a4d50b07d17c725f1dacbbae987d29c7d245aac29bc\": container with ID starting with 7e02bc2145228a6803a90a4d50b07d17c725f1dacbbae987d29c7d245aac29bc not found: ID does not exist" containerID="7e02bc2145228a6803a90a4d50b07d17c725f1dacbbae987d29c7d245aac29bc"
Sep 14 02:53:55.288908 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.288877    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e02bc2145228a6803a90a4d50b07d17c725f1dacbbae987d29c7d245aac29bc"} err="failed to get container status \"7e02bc2145228a6803a90a4d50b07d17c725f1dacbbae987d29c7d245aac29bc\": rpc error: code = NotFound desc = could not find container \"7e02bc2145228a6803a90a4d50b07d17c725f1dacbbae987d29c7d245aac29bc\": container with ID starting with 7e02bc2145228a6803a90a4d50b07d17c725f1dacbbae987d29c7d245aac29bc not found: ID does not exist"
Sep 14 02:53:55.306275 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.306256    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-796c6999d5-m4fs4"]
Sep 14 02:53:55.310838 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.310820    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-796c6999d5-m4fs4"]
Sep 14 02:53:55.742579 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:53:55.742547    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c95fc0c6-b606-4784-a7b1-9a9e4be67357" path="/var/lib/kubelet/pods/c95fc0c6-b606-4784-a7b1-9a9e4be67357/volumes"
Sep 14 02:54:21.375040 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:54:21.375003    2579 generic.go:356] "Generic (PLEG): container finished" podID="a9048354-a3ee-4810-8e08-20b70a5332c2" containerID="f3d3a0cfce0c3f5f0c5584d955ac8d3908e49a71d71bd84bb860c5fde35eb2f7" exitCode=0
Sep 14 02:54:21.375040 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:54:21.375029    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52" event={"ID":"a9048354-a3ee-4810-8e08-20b70a5332c2","Type":"ContainerDied","Data":"f3d3a0cfce0c3f5f0c5584d955ac8d3908e49a71d71bd84bb860c5fde35eb2f7"}
Sep 14 02:54:21.375420 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:54:21.375374    2579 scope.go:117] "RemoveContainer" containerID="f3d3a0cfce0c3f5f0c5584d955ac8d3908e49a71d71bd84bb860c5fde35eb2f7"
Sep 14 02:54:22.380437 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:54:22.380400    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-b4z52" event={"ID":"a9048354-a3ee-4810-8e08-20b70a5332c2","Type":"ContainerStarted","Data":"bc87d4971e109d4d25f19575a6f6f8b384d9154ab8c50a3d441b36753f4104aa"}
Sep 14 02:54:27.398665 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:54:27.398634    2579 generic.go:356] "Generic (PLEG): container finished" podID="8c3d78f2-3b6e-41d6-b747-350c6d5b1e70" containerID="eba09fe36157b25b3e9e040a66cd4b2a62ebf51b64bece521fa9fa7246ec625f" exitCode=0
Sep 14 02:54:27.399065 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:54:27.398693    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt" event={"ID":"8c3d78f2-3b6e-41d6-b747-350c6d5b1e70","Type":"ContainerDied","Data":"eba09fe36157b25b3e9e040a66cd4b2a62ebf51b64bece521fa9fa7246ec625f"}
Sep 14 02:54:27.399065 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:54:27.398997    2579 scope.go:117] "RemoveContainer" containerID="eba09fe36157b25b3e9e040a66cd4b2a62ebf51b64bece521fa9fa7246ec625f"
Sep 14 02:54:28.405104 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:54:28.405072    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ncwzt" event={"ID":"8c3d78f2-3b6e-41d6-b747-350c6d5b1e70","Type":"ContainerStarted","Data":"8b17d48efbba557d06c6ec61535f0a2d78447515de08e3b6caca70f2a493ce73"}
Sep 14 02:57:17.653124 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:17.653099    2579 kubelet.go:1648] "Image garbage collection succeeded"
Sep 14 02:57:31.880309 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:31.880271    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wxnkd"]
Sep 14 02:57:31.882700 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:31.880633    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c95fc0c6-b606-4784-a7b1-9a9e4be67357" containerName="registry"
Sep 14 02:57:31.882700 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:31.880650    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c95fc0c6-b606-4784-a7b1-9a9e4be67357" containerName="registry"
Sep 14 02:57:31.882700 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:31.880739    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c95fc0c6-b606-4784-a7b1-9a9e4be67357" containerName="registry"
Sep 14 02:57:31.883553 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:31.883534    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wxnkd"
Sep 14 02:57:31.886356 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:31.886336    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 14 02:57:31.886452 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:31.886392    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-xxl47\""
Sep 14 02:57:31.887725 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:31.887707    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 14 02:57:31.901563 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:31.901542    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wxnkd"]
Sep 14 02:57:31.962298 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:31.962274    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/742bed19-811d-4dc4-937d-0a5515226a60-bound-sa-token\") pod \"cert-manager-55b4d954d5-wxnkd\" (UID: \"742bed19-811d-4dc4-937d-0a5515226a60\") " pod="cert-manager/cert-manager-55b4d954d5-wxnkd"
Sep 14 02:57:31.962399 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:31.962318    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9flqk\" (UniqueName: \"kubernetes.io/projected/742bed19-811d-4dc4-937d-0a5515226a60-kube-api-access-9flqk\") pod \"cert-manager-55b4d954d5-wxnkd\" (UID: \"742bed19-811d-4dc4-937d-0a5515226a60\") " pod="cert-manager/cert-manager-55b4d954d5-wxnkd"
Sep 14 02:57:32.062912 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:32.062883    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9flqk\" (UniqueName: \"kubernetes.io/projected/742bed19-811d-4dc4-937d-0a5515226a60-kube-api-access-9flqk\") pod \"cert-manager-55b4d954d5-wxnkd\" (UID: \"742bed19-811d-4dc4-937d-0a5515226a60\") " pod="cert-manager/cert-manager-55b4d954d5-wxnkd"
Sep 14 02:57:32.063062 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:32.062941    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/742bed19-811d-4dc4-937d-0a5515226a60-bound-sa-token\") pod \"cert-manager-55b4d954d5-wxnkd\" (UID: \"742bed19-811d-4dc4-937d-0a5515226a60\") " pod="cert-manager/cert-manager-55b4d954d5-wxnkd"
Sep 14 02:57:32.072533 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:32.072512    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9flqk\" (UniqueName: \"kubernetes.io/projected/742bed19-811d-4dc4-937d-0a5515226a60-kube-api-access-9flqk\") pod \"cert-manager-55b4d954d5-wxnkd\" (UID: \"742bed19-811d-4dc4-937d-0a5515226a60\") " pod="cert-manager/cert-manager-55b4d954d5-wxnkd"
Sep 14 02:57:32.072779 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:32.072758    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/742bed19-811d-4dc4-937d-0a5515226a60-bound-sa-token\") pod \"cert-manager-55b4d954d5-wxnkd\" (UID: \"742bed19-811d-4dc4-937d-0a5515226a60\") " pod="cert-manager/cert-manager-55b4d954d5-wxnkd"
Sep 14 02:57:32.192388 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:32.192318    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wxnkd"
Sep 14 02:57:32.467638 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:32.467567    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wxnkd"]
Sep 14 02:57:32.470883 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:57:32.470856    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod742bed19_811d_4dc4_937d_0a5515226a60.slice/crio-c1a45d2d7d76a0729ccc1c40c8bd12e9e71ee6b38fe788ee24b9ca6e6f755429 WatchSource:0}: Error finding container c1a45d2d7d76a0729ccc1c40c8bd12e9e71ee6b38fe788ee24b9ca6e6f755429: Status 404 returned error can't find the container with id c1a45d2d7d76a0729ccc1c40c8bd12e9e71ee6b38fe788ee24b9ca6e6f755429
Sep 14 02:57:32.472849 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:32.472832    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 02:57:33.049109 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:33.049071    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wxnkd" event={"ID":"742bed19-811d-4dc4-937d-0a5515226a60","Type":"ContainerStarted","Data":"c1a45d2d7d76a0729ccc1c40c8bd12e9e71ee6b38fe788ee24b9ca6e6f755429"}
Sep 14 02:57:36.064481 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:36.064443    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wxnkd" event={"ID":"742bed19-811d-4dc4-937d-0a5515226a60","Type":"ContainerStarted","Data":"6fa4c6fd7fcb70e7baccf7642e3acd1419545b93d212b9182e69b9f0de4abb7d"}
Sep 14 02:57:36.084578 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:36.084538    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-wxnkd" podStartSLOduration=2.131274568 podStartE2EDuration="5.084525812s" totalImagesPullingTime="2.953251244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:57:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:57:36.083879727 +0000 UTC m=+318.869941639" watchObservedRunningTime="2026-09-14 02:57:36.084525812 +0000 UTC m=+318.870587725"
Sep 14 02:57:50.493188 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.493141    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"]
Sep 14 02:57:50.496380 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.496365    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:50.502995 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.502968    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 14 02:57:50.502995 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.502993    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 14 02:57:50.503158 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.503046    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-49p7j\""
Sep 14 02:57:50.503158 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.503080    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 14 02:57:50.503495 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.503481    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 14 02:57:50.532362 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.532337    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"]
Sep 14 02:57:50.599598 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.599566    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1a6bb5e4-d156-4977-9b35-249562731e42-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67ffb7699c-85hz6\" (UID: \"1a6bb5e4-d156-4977-9b35-249562731e42\") " pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:50.599747 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.599608    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4md5z\" (UniqueName: \"kubernetes.io/projected/1a6bb5e4-d156-4977-9b35-249562731e42-kube-api-access-4md5z\") pod \"opendatahub-operator-controller-manager-67ffb7699c-85hz6\" (UID: \"1a6bb5e4-d156-4977-9b35-249562731e42\") " pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:50.599747 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.599631    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1a6bb5e4-d156-4977-9b35-249562731e42-manifests\") pod \"opendatahub-operator-controller-manager-67ffb7699c-85hz6\" (UID: \"1a6bb5e4-d156-4977-9b35-249562731e42\") " pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:50.599747 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.599705    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a6bb5e4-d156-4977-9b35-249562731e42-webhook-cert\") pod \"opendatahub-operator-controller-manager-67ffb7699c-85hz6\" (UID: \"1a6bb5e4-d156-4977-9b35-249562731e42\") " pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:50.599747 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.599742    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a6bb5e4-d156-4977-9b35-249562731e42-tmp\") pod \"opendatahub-operator-controller-manager-67ffb7699c-85hz6\" (UID: \"1a6bb5e4-d156-4977-9b35-249562731e42\") " pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:50.700673 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.700645    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1a6bb5e4-d156-4977-9b35-249562731e42-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67ffb7699c-85hz6\" (UID: \"1a6bb5e4-d156-4977-9b35-249562731e42\") " pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:50.700833 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.700681    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4md5z\" (UniqueName: \"kubernetes.io/projected/1a6bb5e4-d156-4977-9b35-249562731e42-kube-api-access-4md5z\") pod \"opendatahub-operator-controller-manager-67ffb7699c-85hz6\" (UID: \"1a6bb5e4-d156-4977-9b35-249562731e42\") " pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:50.700833 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.700698    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1a6bb5e4-d156-4977-9b35-249562731e42-manifests\") pod \"opendatahub-operator-controller-manager-67ffb7699c-85hz6\" (UID: \"1a6bb5e4-d156-4977-9b35-249562731e42\") " pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:50.700833 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.700729    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a6bb5e4-d156-4977-9b35-249562731e42-webhook-cert\") pod \"opendatahub-operator-controller-manager-67ffb7699c-85hz6\" (UID: \"1a6bb5e4-d156-4977-9b35-249562731e42\") " pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:50.700833 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.700748    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a6bb5e4-d156-4977-9b35-249562731e42-tmp\") pod \"opendatahub-operator-controller-manager-67ffb7699c-85hz6\" (UID: \"1a6bb5e4-d156-4977-9b35-249562731e42\") " pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:50.701239 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.701210    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1a6bb5e4-d156-4977-9b35-249562731e42-manifests\") pod \"opendatahub-operator-controller-manager-67ffb7699c-85hz6\" (UID: \"1a6bb5e4-d156-4977-9b35-249562731e42\") " pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:50.703316 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.703291    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a6bb5e4-d156-4977-9b35-249562731e42-tmp\") pod \"opendatahub-operator-controller-manager-67ffb7699c-85hz6\" (UID: \"1a6bb5e4-d156-4977-9b35-249562731e42\") " pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:50.703431 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.703295    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1a6bb5e4-d156-4977-9b35-249562731e42-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67ffb7699c-85hz6\" (UID: \"1a6bb5e4-d156-4977-9b35-249562731e42\") " pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:50.703431 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.703386    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a6bb5e4-d156-4977-9b35-249562731e42-webhook-cert\") pod \"opendatahub-operator-controller-manager-67ffb7699c-85hz6\" (UID: \"1a6bb5e4-d156-4977-9b35-249562731e42\") " pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:50.715776 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.715757    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4md5z\" (UniqueName: \"kubernetes.io/projected/1a6bb5e4-d156-4977-9b35-249562731e42-kube-api-access-4md5z\") pod \"opendatahub-operator-controller-manager-67ffb7699c-85hz6\" (UID: \"1a6bb5e4-d156-4977-9b35-249562731e42\") " pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:50.805686 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:50.805633    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:51.093283 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:51.093208    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"]
Sep 14 02:57:51.096486 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:57:51.096460    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a6bb5e4_d156_4977_9b35_249562731e42.slice/crio-fe30ec966c7bdaab14457a8e6cc9d52b724f91a7d548ffb33236100a6c0a375e WatchSource:0}: Error finding container fe30ec966c7bdaab14457a8e6cc9d52b724f91a7d548ffb33236100a6c0a375e: Status 404 returned error can't find the container with id fe30ec966c7bdaab14457a8e6cc9d52b724f91a7d548ffb33236100a6c0a375e
Sep 14 02:57:51.118013 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:51.117983    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6" event={"ID":"1a6bb5e4-d156-4977-9b35-249562731e42","Type":"ContainerStarted","Data":"fe30ec966c7bdaab14457a8e6cc9d52b724f91a7d548ffb33236100a6c0a375e"}
Sep 14 02:57:54.132918 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:54.132890    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6" event={"ID":"1a6bb5e4-d156-4977-9b35-249562731e42","Type":"ContainerStarted","Data":"e25264dd9620579bd94f2c7a216b716ed97be775fe3f13fc89d9030aeedf5999"}
Sep 14 02:57:55.138399 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:55.138365    2579 generic.go:356] "Generic (PLEG): container finished" podID="1a6bb5e4-d156-4977-9b35-249562731e42" containerID="e25264dd9620579bd94f2c7a216b716ed97be775fe3f13fc89d9030aeedf5999" exitCode=0
Sep 14 02:57:55.138739 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:55.138443    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6" event={"ID":"1a6bb5e4-d156-4977-9b35-249562731e42","Type":"ContainerDied","Data":"e25264dd9620579bd94f2c7a216b716ed97be775fe3f13fc89d9030aeedf5999"}
Sep 14 02:57:56.145511 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:56.145472    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6" event={"ID":"1a6bb5e4-d156-4977-9b35-249562731e42","Type":"ContainerStarted","Data":"249baac5ce8984118fae04d3dad0dd3be5a291f14c7238d955b6c7ec697d87f4"}
Sep 14 02:57:56.145987 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:56.145566    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:57:56.180289 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:57:56.180237    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6" podStartSLOduration=1.855174403 podStartE2EDuration="6.180222423s" totalImagesPullingTime="3.317462184s" totalInitContainerRuntime="1.007585836s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:57:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:57:56.177973049 +0000 UTC m=+338.964034962" watchObservedRunningTime="2026-09-14 02:57:56.180222423 +0000 UTC m=+338.966284335"
Sep 14 02:58:07.151483 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:07.151452    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-67ffb7699c-85hz6"
Sep 14 02:58:10.173083 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.173004    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-798674575b-dp5dl"]
Sep 14 02:58:10.221542 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.221510    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-798674575b-dp5dl"]
Sep 14 02:58:10.221702 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.221609    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-798674575b-dp5dl"
Sep 14 02:58:10.231139 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.231111    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 14 02:58:10.231280 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.231152    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 14 02:58:10.231521 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.231506    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 14 02:58:10.237549 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.237534    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 14 02:58:10.347994 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.347968    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9j77\" (UniqueName: \"kubernetes.io/projected/a43bf689-dde6-4944-a08a-d012c421df06-kube-api-access-j9j77\") pod \"kube-auth-proxy-798674575b-dp5dl\" (UID: \"a43bf689-dde6-4944-a08a-d012c421df06\") " pod="openshift-ingress/kube-auth-proxy-798674575b-dp5dl"
Sep 14 02:58:10.348122 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.348011    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a43bf689-dde6-4944-a08a-d012c421df06-tmp\") pod \"kube-auth-proxy-798674575b-dp5dl\" (UID: \"a43bf689-dde6-4944-a08a-d012c421df06\") " pod="openshift-ingress/kube-auth-proxy-798674575b-dp5dl"
Sep 14 02:58:10.348122 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.348093    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a43bf689-dde6-4944-a08a-d012c421df06-tls-certs\") pod \"kube-auth-proxy-798674575b-dp5dl\" (UID: \"a43bf689-dde6-4944-a08a-d012c421df06\") " pod="openshift-ingress/kube-auth-proxy-798674575b-dp5dl"
Sep 14 02:58:10.449282 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.449220    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a43bf689-dde6-4944-a08a-d012c421df06-tls-certs\") pod \"kube-auth-proxy-798674575b-dp5dl\" (UID: \"a43bf689-dde6-4944-a08a-d012c421df06\") " pod="openshift-ingress/kube-auth-proxy-798674575b-dp5dl"
Sep 14 02:58:10.449407 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.449282    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9j77\" (UniqueName: \"kubernetes.io/projected/a43bf689-dde6-4944-a08a-d012c421df06-kube-api-access-j9j77\") pod \"kube-auth-proxy-798674575b-dp5dl\" (UID: \"a43bf689-dde6-4944-a08a-d012c421df06\") " pod="openshift-ingress/kube-auth-proxy-798674575b-dp5dl"
Sep 14 02:58:10.449407 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.449314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a43bf689-dde6-4944-a08a-d012c421df06-tmp\") pod \"kube-auth-proxy-798674575b-dp5dl\" (UID: \"a43bf689-dde6-4944-a08a-d012c421df06\") " pod="openshift-ingress/kube-auth-proxy-798674575b-dp5dl"
Sep 14 02:58:10.449407 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:58:10.449372    2579 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 14 02:58:10.449526 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:58:10.449440    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a43bf689-dde6-4944-a08a-d012c421df06-tls-certs podName:a43bf689-dde6-4944-a08a-d012c421df06 nodeName:}" failed. No retries permitted until 2026-09-14 02:58:10.949420449 +0000 UTC m=+353.735482353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a43bf689-dde6-4944-a08a-d012c421df06-tls-certs") pod "kube-auth-proxy-798674575b-dp5dl" (UID: "a43bf689-dde6-4944-a08a-d012c421df06") : secret "kube-auth-proxy-tls" not found
Sep 14 02:58:10.451711 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.451686    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a43bf689-dde6-4944-a08a-d012c421df06-tmp\") pod \"kube-auth-proxy-798674575b-dp5dl\" (UID: \"a43bf689-dde6-4944-a08a-d012c421df06\") " pod="openshift-ingress/kube-auth-proxy-798674575b-dp5dl"
Sep 14 02:58:10.489582 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.489562    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9j77\" (UniqueName: \"kubernetes.io/projected/a43bf689-dde6-4944-a08a-d012c421df06-kube-api-access-j9j77\") pod \"kube-auth-proxy-798674575b-dp5dl\" (UID: \"a43bf689-dde6-4944-a08a-d012c421df06\") " pod="openshift-ingress/kube-auth-proxy-798674575b-dp5dl"
Sep 14 02:58:10.953064 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.953032    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a43bf689-dde6-4944-a08a-d012c421df06-tls-certs\") pod \"kube-auth-proxy-798674575b-dp5dl\" (UID: \"a43bf689-dde6-4944-a08a-d012c421df06\") " pod="openshift-ingress/kube-auth-proxy-798674575b-dp5dl"
Sep 14 02:58:10.955508 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:10.955480    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a43bf689-dde6-4944-a08a-d012c421df06-tls-certs\") pod \"kube-auth-proxy-798674575b-dp5dl\" (UID: \"a43bf689-dde6-4944-a08a-d012c421df06\") " pod="openshift-ingress/kube-auth-proxy-798674575b-dp5dl"
Sep 14 02:58:11.131077 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:11.131042    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-798674575b-dp5dl"
Sep 14 02:58:11.461460 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:58:11.461435    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda43bf689_dde6_4944_a08a_d012c421df06.slice/crio-73c8ebf8d501877c33fea223a22f3539d3d96cf399d99a4c6af439f0b91b8e81 WatchSource:0}: Error finding container 73c8ebf8d501877c33fea223a22f3539d3d96cf399d99a4c6af439f0b91b8e81: Status 404 returned error can't find the container with id 73c8ebf8d501877c33fea223a22f3539d3d96cf399d99a4c6af439f0b91b8e81
Sep 14 02:58:11.465153 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:11.465126    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-798674575b-dp5dl"]
Sep 14 02:58:12.210981 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:12.210873    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-798674575b-dp5dl" event={"ID":"a43bf689-dde6-4944-a08a-d012c421df06","Type":"ContainerStarted","Data":"73c8ebf8d501877c33fea223a22f3539d3d96cf399d99a4c6af439f0b91b8e81"}
Sep 14 02:58:13.283049 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.283014    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"]
Sep 14 02:58:13.286708 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.286686    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.292626 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.292571    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 14 02:58:13.292626 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.292596    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 14 02:58:13.292837 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.292825    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hqds2\""
Sep 14 02:58:13.293037 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.293018    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 14 02:58:13.294789 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.294758    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 02:58:13.317424 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.317400    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"]
Sep 14 02:58:13.475992 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.475960    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/52268d48-fdf4-43f8-a64d-63b5dc9e4894-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.476153 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.475998    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.476153 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.476031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.476153 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.476058    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.476153 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.476078    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdmvk\" (UniqueName: \"kubernetes.io/projected/52268d48-fdf4-43f8-a64d-63b5dc9e4894-kube-api-access-xdmvk\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.476332 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.476153    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.476332 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.476250    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.577508 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.577412    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.577665 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.577525    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/52268d48-fdf4-43f8-a64d-63b5dc9e4894-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.577665 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.577548    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.577665 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.577576    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.577665 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.577611    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.577665 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.577636    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdmvk\" (UniqueName: \"kubernetes.io/projected/52268d48-fdf4-43f8-a64d-63b5dc9e4894-kube-api-access-xdmvk\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.577665 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.577663    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.578620 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.578566    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.580520 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.580408    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/52268d48-fdf4-43f8-a64d-63b5dc9e4894-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.580656 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.580581    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.580656 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.580646    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.580879 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.580859    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.595089 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.595058    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdmvk\" (UniqueName: \"kubernetes.io/projected/52268d48-fdf4-43f8-a64d-63b5dc9e4894-kube-api-access-xdmvk\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.595346 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.595323    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-n7z45\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:13.599232 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:13.599213    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:14.220640 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:14.220606    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-798674575b-dp5dl" event={"ID":"a43bf689-dde6-4944-a08a-d012c421df06","Type":"ContainerStarted","Data":"34ef1b76b4f89d7ec8fab6b6d34ec2f2a260a4e93de55ce2eff58d312200ffa2"}
Sep 14 02:58:14.313120 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:14.313024    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-798674575b-dp5dl" podStartSLOduration=1.691814299 podStartE2EDuration="4.313006901s" totalImagesPullingTime="2.621192602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:58:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:58:14.311884018 +0000 UTC m=+357.097945932" watchObservedRunningTime="2026-09-14 02:58:14.313006901 +0000 UTC m=+357.099068814"
Sep 14 02:58:14.383328 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:14.383303    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"]
Sep 14 02:58:14.384755 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:58:14.384729    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52268d48_fdf4_43f8_a64d_63b5dc9e4894.slice/crio-992edd916b6b0893c05f273867f551c4ba7d9750468f17afa57682a8eb255790 WatchSource:0}: Error finding container 992edd916b6b0893c05f273867f551c4ba7d9750468f17afa57682a8eb255790: Status 404 returned error can't find the container with id 992edd916b6b0893c05f273867f551c4ba7d9750468f17afa57682a8eb255790
Sep 14 02:58:15.226799 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:15.226761    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45" event={"ID":"52268d48-fdf4-43f8-a64d-63b5dc9e4894","Type":"ContainerStarted","Data":"992edd916b6b0893c05f273867f551c4ba7d9750468f17afa57682a8eb255790"}
Sep 14 02:58:16.919022 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:16.918987    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 02:58:16.919281 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:16.919053    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 02:58:17.236903 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:17.236822    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45" event={"ID":"52268d48-fdf4-43f8-a64d-63b5dc9e4894","Type":"ContainerStarted","Data":"93e42f1f860fe061056653a5c66cfbd72c925e3597fe20798810f6d2a72578bf"}
Sep 14 02:58:17.237064 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:17.237045    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:17.238622 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:17.238598    2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-n7z45 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 02:58:17.238731 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:17.238644    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45" podUID="52268d48-fdf4-43f8-a64d-63b5dc9e4894" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 02:58:17.269159 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:17.269104    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45" podStartSLOduration=1.737599292 podStartE2EDuration="4.269087188s" totalImagesPullingTime="2.531487896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:58:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:58:17.268122266 +0000 UTC m=+360.054184184" watchObservedRunningTime="2026-09-14 02:58:17.269087188 +0000 UTC m=+360.055149100"
Sep 14 02:58:18.241207 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:18.241158    2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-n7z45 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 02:58:18.241662 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:18.241237    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45" podUID="52268d48-fdf4-43f8-a64d-63b5dc9e4894" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 02:58:21.242327 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:21.242294    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:21.468774 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:21.468739    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4724d"]
Sep 14 02:58:21.473522 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:21.473506    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4724d"
Sep 14 02:58:21.476988 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:21.476966    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 14 02:58:21.477109 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:21.477058    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-sgtnq\""
Sep 14 02:58:21.486920 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:21.486898    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4724d"]
Sep 14 02:58:21.538575 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:21.538470    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/163a441e-6c6a-4a9c-a372-40d4d7050af3-tls-certs\") pod \"model-serving-api-54d89bd5fc-4724d\" (UID: \"163a441e-6c6a-4a9c-a372-40d4d7050af3\") " pod="opendatahub/model-serving-api-54d89bd5fc-4724d"
Sep 14 02:58:21.538575 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:21.538523    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp6td\" (UniqueName: \"kubernetes.io/projected/163a441e-6c6a-4a9c-a372-40d4d7050af3-kube-api-access-sp6td\") pod \"model-serving-api-54d89bd5fc-4724d\" (UID: \"163a441e-6c6a-4a9c-a372-40d4d7050af3\") " pod="opendatahub/model-serving-api-54d89bd5fc-4724d"
Sep 14 02:58:21.639534 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:21.639500    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/163a441e-6c6a-4a9c-a372-40d4d7050af3-tls-certs\") pod \"model-serving-api-54d89bd5fc-4724d\" (UID: \"163a441e-6c6a-4a9c-a372-40d4d7050af3\") " pod="opendatahub/model-serving-api-54d89bd5fc-4724d"
Sep 14 02:58:21.639534 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:21.639535    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp6td\" (UniqueName: \"kubernetes.io/projected/163a441e-6c6a-4a9c-a372-40d4d7050af3-kube-api-access-sp6td\") pod \"model-serving-api-54d89bd5fc-4724d\" (UID: \"163a441e-6c6a-4a9c-a372-40d4d7050af3\") " pod="opendatahub/model-serving-api-54d89bd5fc-4724d"
Sep 14 02:58:21.642194 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:21.642154    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/163a441e-6c6a-4a9c-a372-40d4d7050af3-tls-certs\") pod \"model-serving-api-54d89bd5fc-4724d\" (UID: \"163a441e-6c6a-4a9c-a372-40d4d7050af3\") " pod="opendatahub/model-serving-api-54d89bd5fc-4724d"
Sep 14 02:58:21.653318 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:21.653297    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp6td\" (UniqueName: \"kubernetes.io/projected/163a441e-6c6a-4a9c-a372-40d4d7050af3-kube-api-access-sp6td\") pod \"model-serving-api-54d89bd5fc-4724d\" (UID: \"163a441e-6c6a-4a9c-a372-40d4d7050af3\") " pod="opendatahub/model-serving-api-54d89bd5fc-4724d"
Sep 14 02:58:21.784460 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:21.784431    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4724d"
Sep 14 02:58:22.071499 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:22.071474    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4724d"]
Sep 14 02:58:22.073194 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:58:22.073145    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod163a441e_6c6a_4a9c_a372_40d4d7050af3.slice/crio-4ec47f4f66a52597807e068712a08282ccb963a56fd9bf3b7105ee05c9b1d128 WatchSource:0}: Error finding container 4ec47f4f66a52597807e068712a08282ccb963a56fd9bf3b7105ee05c9b1d128: Status 404 returned error can't find the container with id 4ec47f4f66a52597807e068712a08282ccb963a56fd9bf3b7105ee05c9b1d128
Sep 14 02:58:22.257948 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:22.257902    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4724d" event={"ID":"163a441e-6c6a-4a9c-a372-40d4d7050af3","Type":"ContainerStarted","Data":"4ec47f4f66a52597807e068712a08282ccb963a56fd9bf3b7105ee05c9b1d128"}
Sep 14 02:58:24.268720 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:24.268683    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4724d" event={"ID":"163a441e-6c6a-4a9c-a372-40d4d7050af3","Type":"ContainerStarted","Data":"e3e5b502843b61b6ca70bca6fa9abe206aa8243c4d5ea3e26d22e482c95e9ec2"}
Sep 14 02:58:24.269085 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:24.268811    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-4724d"
Sep 14 02:58:24.287320 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:24.287276    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-4724d" podStartSLOduration=1.7825208030000002 podStartE2EDuration="3.287263013s" totalImagesPullingTime="1.50474221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:58:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:58:24.286020024 +0000 UTC m=+367.072081938" watchObservedRunningTime="2026-09-14 02:58:24.287263013 +0000 UTC m=+367.073324926"
Sep 14 02:58:28.320744 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.320704    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"]
Sep 14 02:58:28.325094 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.325066    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.348237 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.348208    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"]
Sep 14 02:58:28.389806 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.389776    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c56b5cb9-d498-4d61-b120-c30d5faef3a9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.389956 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.389811    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c56b5cb9-d498-4d61-b120-c30d5faef3a9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.389956 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.389829    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c56b5cb9-d498-4d61-b120-c30d5faef3a9-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.389956 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.389870    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2phw8\" (UniqueName: \"kubernetes.io/projected/c56b5cb9-d498-4d61-b120-c30d5faef3a9-kube-api-access-2phw8\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.389956 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.389918    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c56b5cb9-d498-4d61-b120-c30d5faef3a9-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.390106 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.389978    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c56b5cb9-d498-4d61-b120-c30d5faef3a9-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.390106 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.389999    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c56b5cb9-d498-4d61-b120-c30d5faef3a9-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.491929 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.491889    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c56b5cb9-d498-4d61-b120-c30d5faef3a9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.492119 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.491942    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c56b5cb9-d498-4d61-b120-c30d5faef3a9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.492119 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.491969    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c56b5cb9-d498-4d61-b120-c30d5faef3a9-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.492119 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.491999    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2phw8\" (UniqueName: \"kubernetes.io/projected/c56b5cb9-d498-4d61-b120-c30d5faef3a9-kube-api-access-2phw8\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.492119 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.492021    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c56b5cb9-d498-4d61-b120-c30d5faef3a9-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.492119 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.492068    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c56b5cb9-d498-4d61-b120-c30d5faef3a9-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.492119 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.492099    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c56b5cb9-d498-4d61-b120-c30d5faef3a9-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.492729 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.492698    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c56b5cb9-d498-4d61-b120-c30d5faef3a9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.494857 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.494824    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c56b5cb9-d498-4d61-b120-c30d5faef3a9-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.495020 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.494993    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c56b5cb9-d498-4d61-b120-c30d5faef3a9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.495114 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.495094    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c56b5cb9-d498-4d61-b120-c30d5faef3a9-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.495187 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.495116    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c56b5cb9-d498-4d61-b120-c30d5faef3a9-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.509504 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.509483    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c56b5cb9-d498-4d61-b120-c30d5faef3a9-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.519042 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.519018    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2phw8\" (UniqueName: \"kubernetes.io/projected/c56b5cb9-d498-4d61-b120-c30d5faef3a9-kube-api-access-2phw8\") pod \"istiod-openshift-gateway-5677ccb49b-9kzn4\" (UID: \"c56b5cb9-d498-4d61-b120-c30d5faef3a9\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.634211 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.634127    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:28.936499 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.936475    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"]
Sep 14 02:58:28.937741 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:58:28.937721    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc56b5cb9_d498_4d61_b120_c30d5faef3a9.slice/crio-24458c2110f413acae052650058eaf57f86748135a837c7c390e349e52dfc2a4 WatchSource:0}: Error finding container 24458c2110f413acae052650058eaf57f86748135a837c7c390e349e52dfc2a4: Status 404 returned error can't find the container with id 24458c2110f413acae052650058eaf57f86748135a837c7c390e349e52dfc2a4
Sep 14 02:58:28.939847 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.939822    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 02:58:28.939901 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:28.939876    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 02:58:29.289960 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:29.289915    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4" event={"ID":"c56b5cb9-d498-4d61-b120-c30d5faef3a9","Type":"ContainerStarted","Data":"772acd14d915365fb53ca0dabf31fbe8434638ab8fac4688438d683222a15992"}
Sep 14 02:58:29.290138 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:29.289976    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:29.290138 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:29.289991    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4" event={"ID":"c56b5cb9-d498-4d61-b120-c30d5faef3a9","Type":"ContainerStarted","Data":"24458c2110f413acae052650058eaf57f86748135a837c7c390e349e52dfc2a4"}
Sep 14 02:58:29.291630 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:29.291605    2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-9kzn4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 02:58:29.291749 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:29.291660    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4" podUID="c56b5cb9-d498-4d61-b120-c30d5faef3a9" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 02:58:29.338794 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:29.338749    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4" podStartSLOduration=1.338735006 podStartE2EDuration="1.338735006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:58:29.337827014 +0000 UTC m=+372.123888939" watchObservedRunningTime="2026-09-14 02:58:29.338735006 +0000 UTC m=+372.124796918"
Sep 14 02:58:30.294522 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.294490    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9kzn4"
Sep 14 02:58:30.438169 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.438137    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"]
Sep 14 02:58:30.438615 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.438375    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45" podUID="52268d48-fdf4-43f8-a64d-63b5dc9e4894" containerName="discovery" containerID="cri-o://93e42f1f860fe061056653a5c66cfbd72c925e3597fe20798810f6d2a72578bf" gracePeriod=30
Sep 14 02:58:30.672337 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.672314    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:30.707979 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.707956    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-token\") pod \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") "
Sep 14 02:58:30.708064 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.708005    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-csr-ca-configmap\") pod \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") "
Sep 14 02:58:30.708064 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.708033    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-cacerts\") pod \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") "
Sep 14 02:58:30.708166 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.708141    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-csr-dns-cert\") pod \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") "
Sep 14 02:58:30.708250 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.708221    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdmvk\" (UniqueName: \"kubernetes.io/projected/52268d48-fdf4-43f8-a64d-63b5dc9e4894-kube-api-access-xdmvk\") pod \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") "
Sep 14 02:58:30.708304 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.708256    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-kubeconfig\") pod \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") "
Sep 14 02:58:30.708357 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.708306    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/52268d48-fdf4-43f8-a64d-63b5dc9e4894-local-certs\") pod \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\" (UID: \"52268d48-fdf4-43f8-a64d-63b5dc9e4894\") "
Sep 14 02:58:30.708434 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.708411    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "52268d48-fdf4-43f8-a64d-63b5dc9e4894" (UID: "52268d48-fdf4-43f8-a64d-63b5dc9e4894"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 02:58:30.708643 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.708622    2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-csr-ca-configmap\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 02:58:30.710595 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.710568    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "52268d48-fdf4-43f8-a64d-63b5dc9e4894" (UID: "52268d48-fdf4-43f8-a64d-63b5dc9e4894"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:58:30.710595 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.710581    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52268d48-fdf4-43f8-a64d-63b5dc9e4894-kube-api-access-xdmvk" (OuterVolumeSpecName: "kube-api-access-xdmvk") pod "52268d48-fdf4-43f8-a64d-63b5dc9e4894" (UID: "52268d48-fdf4-43f8-a64d-63b5dc9e4894"). InnerVolumeSpecName "kube-api-access-xdmvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:58:30.710843 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.710817    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-token" (OuterVolumeSpecName: "istio-token") pod "52268d48-fdf4-43f8-a64d-63b5dc9e4894" (UID: "52268d48-fdf4-43f8-a64d-63b5dc9e4894"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 02:58:30.710946 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.710926    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "52268d48-fdf4-43f8-a64d-63b5dc9e4894" (UID: "52268d48-fdf4-43f8-a64d-63b5dc9e4894"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:58:30.711003 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.710943    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-cacerts" (OuterVolumeSpecName: "cacerts") pod "52268d48-fdf4-43f8-a64d-63b5dc9e4894" (UID: "52268d48-fdf4-43f8-a64d-63b5dc9e4894"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 02:58:30.711003 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.710964    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52268d48-fdf4-43f8-a64d-63b5dc9e4894-local-certs" (OuterVolumeSpecName: "local-certs") pod "52268d48-fdf4-43f8-a64d-63b5dc9e4894" (UID: "52268d48-fdf4-43f8-a64d-63b5dc9e4894"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 02:58:30.810092 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.810032    2579 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/52268d48-fdf4-43f8-a64d-63b5dc9e4894-local-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 02:58:30.810092 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.810053    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 02:58:30.810092 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.810062    2579 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-cacerts\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 02:58:30.810092 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.810070    2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-csr-dns-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 02:58:30.810092 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.810079    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdmvk\" (UniqueName: \"kubernetes.io/projected/52268d48-fdf4-43f8-a64d-63b5dc9e4894-kube-api-access-xdmvk\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 02:58:30.810092 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:30.810087    2579 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/52268d48-fdf4-43f8-a64d-63b5dc9e4894-istio-kubeconfig\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 02:58:31.299924 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:31.299889    2579 generic.go:356] "Generic (PLEG): container finished" podID="52268d48-fdf4-43f8-a64d-63b5dc9e4894" containerID="93e42f1f860fe061056653a5c66cfbd72c925e3597fe20798810f6d2a72578bf" exitCode=0
Sep 14 02:58:31.300108 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:31.299946    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"
Sep 14 02:58:31.300108 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:31.299984    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45" event={"ID":"52268d48-fdf4-43f8-a64d-63b5dc9e4894","Type":"ContainerDied","Data":"93e42f1f860fe061056653a5c66cfbd72c925e3597fe20798810f6d2a72578bf"}
Sep 14 02:58:31.300108 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:31.300035    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45" event={"ID":"52268d48-fdf4-43f8-a64d-63b5dc9e4894","Type":"ContainerDied","Data":"992edd916b6b0893c05f273867f551c4ba7d9750468f17afa57682a8eb255790"}
Sep 14 02:58:31.300108 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:31.300055    2579 scope.go:117] "RemoveContainer" containerID="93e42f1f860fe061056653a5c66cfbd72c925e3597fe20798810f6d2a72578bf"
Sep 14 02:58:31.309696 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:31.309679    2579 scope.go:117] "RemoveContainer" containerID="93e42f1f860fe061056653a5c66cfbd72c925e3597fe20798810f6d2a72578bf"
Sep 14 02:58:31.309997 ip-10-0-138-244 kubenswrapper[2579]: E0914 02:58:31.309980    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93e42f1f860fe061056653a5c66cfbd72c925e3597fe20798810f6d2a72578bf\": container with ID starting with 93e42f1f860fe061056653a5c66cfbd72c925e3597fe20798810f6d2a72578bf not found: ID does not exist" containerID="93e42f1f860fe061056653a5c66cfbd72c925e3597fe20798810f6d2a72578bf"
Sep 14 02:58:31.310070 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:31.310004    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93e42f1f860fe061056653a5c66cfbd72c925e3597fe20798810f6d2a72578bf"} err="failed to get container status \"93e42f1f860fe061056653a5c66cfbd72c925e3597fe20798810f6d2a72578bf\": rpc error: code = NotFound desc = could not find container \"93e42f1f860fe061056653a5c66cfbd72c925e3597fe20798810f6d2a72578bf\": container with ID starting with 93e42f1f860fe061056653a5c66cfbd72c925e3597fe20798810f6d2a72578bf not found: ID does not exist"
Sep 14 02:58:31.346835 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:31.346810    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"]
Sep 14 02:58:31.358403 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:31.358380    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n7z45"]
Sep 14 02:58:31.747046 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:31.747011    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52268d48-fdf4-43f8-a64d-63b5dc9e4894" path="/var/lib/kubelet/pods/52268d48-fdf4-43f8-a64d-63b5dc9e4894/volumes"
Sep 14 02:58:32.924775 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:32.924745    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-252ng"]
Sep 14 02:58:32.925144 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:32.925032    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52268d48-fdf4-43f8-a64d-63b5dc9e4894" containerName="discovery"
Sep 14 02:58:32.925144 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:32.925044    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="52268d48-fdf4-43f8-a64d-63b5dc9e4894" containerName="discovery"
Sep 14 02:58:32.925144 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:32.925129    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="52268d48-fdf4-43f8-a64d-63b5dc9e4894" containerName="discovery"
Sep 14 02:58:32.929342 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:32.929326    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-252ng"
Sep 14 02:58:32.963366 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:32.963348    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 14 02:58:32.964785 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:32.964747    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-zhxtr\""
Sep 14 02:58:33.028629 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:33.028594    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c76688b-dff6-4370-970e-68125a400067-cert\") pod \"kserve-controller-manager-d6f4b65b8-252ng\" (UID: \"8c76688b-dff6-4370-970e-68125a400067\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-252ng"
Sep 14 02:58:33.028790 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:33.028641    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj2td\" (UniqueName: \"kubernetes.io/projected/8c76688b-dff6-4370-970e-68125a400067-kube-api-access-gj2td\") pod \"kserve-controller-manager-d6f4b65b8-252ng\" (UID: \"8c76688b-dff6-4370-970e-68125a400067\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-252ng"
Sep 14 02:58:33.033791 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:33.033763    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-252ng"]
Sep 14 02:58:33.129360 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:33.129319    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c76688b-dff6-4370-970e-68125a400067-cert\") pod \"kserve-controller-manager-d6f4b65b8-252ng\" (UID: \"8c76688b-dff6-4370-970e-68125a400067\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-252ng"
Sep 14 02:58:33.129360 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:33.129370    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj2td\" (UniqueName: \"kubernetes.io/projected/8c76688b-dff6-4370-970e-68125a400067-kube-api-access-gj2td\") pod \"kserve-controller-manager-d6f4b65b8-252ng\" (UID: \"8c76688b-dff6-4370-970e-68125a400067\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-252ng"
Sep 14 02:58:33.131960 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:33.131928    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c76688b-dff6-4370-970e-68125a400067-cert\") pod \"kserve-controller-manager-d6f4b65b8-252ng\" (UID: \"8c76688b-dff6-4370-970e-68125a400067\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-252ng"
Sep 14 02:58:33.149800 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:33.149770    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj2td\" (UniqueName: \"kubernetes.io/projected/8c76688b-dff6-4370-970e-68125a400067-kube-api-access-gj2td\") pod \"kserve-controller-manager-d6f4b65b8-252ng\" (UID: \"8c76688b-dff6-4370-970e-68125a400067\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-252ng"
Sep 14 02:58:33.238680 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:33.238616    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-252ng"
Sep 14 02:58:33.542648 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:33.542621    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-252ng"]
Sep 14 02:58:33.545659 ip-10-0-138-244 kubenswrapper[2579]: W0914 02:58:33.545634    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c76688b_dff6_4370_970e_68125a400067.slice/crio-39e2533b0691f584e96cbb2eafc68a51cf82d8bb347ee6d935e9da9c74882612 WatchSource:0}: Error finding container 39e2533b0691f584e96cbb2eafc68a51cf82d8bb347ee6d935e9da9c74882612: Status 404 returned error can't find the container with id 39e2533b0691f584e96cbb2eafc68a51cf82d8bb347ee6d935e9da9c74882612
Sep 14 02:58:34.315429 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:34.315385    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-252ng" event={"ID":"8c76688b-dff6-4370-970e-68125a400067","Type":"ContainerStarted","Data":"39e2533b0691f584e96cbb2eafc68a51cf82d8bb347ee6d935e9da9c74882612"}
Sep 14 02:58:35.277659 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:35.277624    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-4724d"
Sep 14 02:58:36.325898 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:36.325861    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-252ng" event={"ID":"8c76688b-dff6-4370-970e-68125a400067","Type":"ContainerStarted","Data":"94a234858179015ff61cbf5830412ff2685451cde0e43d154788e8d463ee8263"}
Sep 14 02:58:36.326329 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:36.325981    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-252ng"
Sep 14 02:58:36.351986 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:58:36.351935    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-252ng" podStartSLOduration=2.304018734 podStartE2EDuration="4.351920002s" totalImagesPullingTime="2.047901268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 02:58:36.35120375 +0000 UTC m=+379.137265654" watchObservedRunningTime="2026-09-14 02:58:36.351920002 +0000 UTC m=+379.137981915"
Sep 14 02:59:07.336496 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:59:07.336466    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-252ng"
Sep 14 02:59:59.497795 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:59:59.497763    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-vqfvm"]
Sep 14 02:59:59.501055 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:59:59.501038    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-vqfvm"
Sep 14 02:59:59.504469 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:59:59.504444    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 14 02:59:59.504575 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:59:59.504532    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-9f598\""
Sep 14 02:59:59.505427 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:59:59.505410    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 14 02:59:59.510878 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:59:59.510858    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-vqfvm"]
Sep 14 02:59:59.596776 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:59:59.596754    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pvgl\" (UniqueName: \"kubernetes.io/projected/4f2de835-c0c9-4820-b7c2-05a64373675e-kube-api-access-4pvgl\") pod \"authorino-operator-76b478bd78-vqfvm\" (UID: \"4f2de835-c0c9-4820-b7c2-05a64373675e\") " pod="kuadrant-system/authorino-operator-76b478bd78-vqfvm"
Sep 14 02:59:59.697483 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:59:59.697463    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pvgl\" (UniqueName: \"kubernetes.io/projected/4f2de835-c0c9-4820-b7c2-05a64373675e-kube-api-access-4pvgl\") pod \"authorino-operator-76b478bd78-vqfvm\" (UID: \"4f2de835-c0c9-4820-b7c2-05a64373675e\") " pod="kuadrant-system/authorino-operator-76b478bd78-vqfvm"
Sep 14 02:59:59.713803 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:59:59.713771    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pvgl\" (UniqueName: \"kubernetes.io/projected/4f2de835-c0c9-4820-b7c2-05a64373675e-kube-api-access-4pvgl\") pod \"authorino-operator-76b478bd78-vqfvm\" (UID: \"4f2de835-c0c9-4820-b7c2-05a64373675e\") " pod="kuadrant-system/authorino-operator-76b478bd78-vqfvm"
Sep 14 02:59:59.811774 ip-10-0-138-244 kubenswrapper[2579]: I0914 02:59:59.811717    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-vqfvm"
Sep 14 03:00:00.089860 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:00.089834    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-vqfvm"]
Sep 14 03:00:00.091671 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:00:00.091637    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f2de835_c0c9_4820_b7c2_05a64373675e.slice/crio-b6139f61f04bc3333be015ccdc3178a3579def592e524ddb7ee214ad10c8f03d WatchSource:0}: Error finding container b6139f61f04bc3333be015ccdc3178a3579def592e524ddb7ee214ad10c8f03d: Status 404 returned error can't find the container with id b6139f61f04bc3333be015ccdc3178a3579def592e524ddb7ee214ad10c8f03d
Sep 14 03:00:00.683304 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:00.683272    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-vqfvm" event={"ID":"4f2de835-c0c9-4820-b7c2-05a64373675e","Type":"ContainerStarted","Data":"b6139f61f04bc3333be015ccdc3178a3579def592e524ddb7ee214ad10c8f03d"}
Sep 14 03:00:03.701310 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:03.701278    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-vqfvm" event={"ID":"4f2de835-c0c9-4820-b7c2-05a64373675e","Type":"ContainerStarted","Data":"94017cca4c06c2670d3860e4d547b5fa71bdcfa76ed5e1fd0da3f3008c4aaa0d"}
Sep 14 03:00:03.701727 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:03.701379    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-vqfvm"
Sep 14 03:00:03.725786 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:03.725742    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-vqfvm" podStartSLOduration=2.014118343 podStartE2EDuration="4.725728222s" totalImagesPullingTime="2.711609879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 02:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:00:03.723445288 +0000 UTC m=+466.509507199" watchObservedRunningTime="2026-09-14 03:00:03.725728222 +0000 UTC m=+466.511790135"
Sep 14 03:00:14.708351 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:14.708320    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-vqfvm"
Sep 14 03:00:29.906533 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:29.906482    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"]
Sep 14 03:00:29.911133 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:29.911111    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"
Sep 14 03:00:29.915567 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:29.915530    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8sprk\""
Sep 14 03:00:29.956051 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:29.956024    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"]
Sep 14 03:00:30.029965 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:30.029936    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eaf38a4b-1bf6-43c3-9c09-489a472f1fd2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-44gtm\" (UID: \"eaf38a4b-1bf6-43c3-9c09-489a472f1fd2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"
Sep 14 03:00:30.029965 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:30.029966    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6nmj\" (UniqueName: \"kubernetes.io/projected/eaf38a4b-1bf6-43c3-9c09-489a472f1fd2-kube-api-access-k6nmj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-44gtm\" (UID: \"eaf38a4b-1bf6-43c3-9c09-489a472f1fd2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"
Sep 14 03:00:30.130777 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:30.130741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eaf38a4b-1bf6-43c3-9c09-489a472f1fd2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-44gtm\" (UID: \"eaf38a4b-1bf6-43c3-9c09-489a472f1fd2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"
Sep 14 03:00:30.130777 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:30.130781    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6nmj\" (UniqueName: \"kubernetes.io/projected/eaf38a4b-1bf6-43c3-9c09-489a472f1fd2-kube-api-access-k6nmj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-44gtm\" (UID: \"eaf38a4b-1bf6-43c3-9c09-489a472f1fd2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"
Sep 14 03:00:30.131098 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:30.131080    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eaf38a4b-1bf6-43c3-9c09-489a472f1fd2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-44gtm\" (UID: \"eaf38a4b-1bf6-43c3-9c09-489a472f1fd2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"
Sep 14 03:00:30.145377 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:30.145351    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6nmj\" (UniqueName: \"kubernetes.io/projected/eaf38a4b-1bf6-43c3-9c09-489a472f1fd2-kube-api-access-k6nmj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-44gtm\" (UID: \"eaf38a4b-1bf6-43c3-9c09-489a472f1fd2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"
Sep 14 03:00:30.221469 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:30.221412    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"
Sep 14 03:00:30.516203 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:30.516066    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"]
Sep 14 03:00:30.519087 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:00:30.519055    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaf38a4b_1bf6_43c3_9c09_489a472f1fd2.slice/crio-f8f1d4370d8de3e1846cfd0ad6a98054d037b7d3582f03f310b51361f6826c0f WatchSource:0}: Error finding container f8f1d4370d8de3e1846cfd0ad6a98054d037b7d3582f03f310b51361f6826c0f: Status 404 returned error can't find the container with id f8f1d4370d8de3e1846cfd0ad6a98054d037b7d3582f03f310b51361f6826c0f
Sep 14 03:00:30.810164 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:30.810085    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm" event={"ID":"eaf38a4b-1bf6-43c3-9c09-489a472f1fd2","Type":"ContainerStarted","Data":"f8f1d4370d8de3e1846cfd0ad6a98054d037b7d3582f03f310b51361f6826c0f"}
Sep 14 03:00:35.836268 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:35.836231    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm" event={"ID":"eaf38a4b-1bf6-43c3-9c09-489a472f1fd2","Type":"ContainerStarted","Data":"88fa8d06dec43408d48ad37b302782aa09d56ab0044ba0bf0d2a8889b205df66"}
Sep 14 03:00:35.836695 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:35.836385    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"
Sep 14 03:00:35.869125 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:35.869074    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm" podStartSLOduration=2.470497056 podStartE2EDuration="6.869059428s" totalImagesPullingTime="4.398562372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:00:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:00:35.865938509 +0000 UTC m=+498.652000421" watchObservedRunningTime="2026-09-14 03:00:35.869059428 +0000 UTC m=+498.655121343"
Sep 14 03:00:46.843193 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:00:46.843152    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"
Sep 14 03:01:34.544594 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:01:34.544556    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66b54f7d67-rdqzr"]
Sep 14 03:01:34.547161 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:01:34.547139    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66b54f7d67-rdqzr"
Sep 14 03:01:34.550861 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:01:34.550837    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 14 03:01:34.550954 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:01:34.550837    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9z7kg\""
Sep 14 03:01:34.557890 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:01:34.557869    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66b54f7d67-rdqzr"]
Sep 14 03:01:34.620994 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:01:34.620964    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dc73575f-364a-4eb5-8f53-3aba367f798f-tls-cert\") pod \"authorino-66b54f7d67-rdqzr\" (UID: \"dc73575f-364a-4eb5-8f53-3aba367f798f\") " pod="kuadrant-system/authorino-66b54f7d67-rdqzr"
Sep 14 03:01:34.621115 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:01:34.621019    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82pjw\" (UniqueName: \"kubernetes.io/projected/dc73575f-364a-4eb5-8f53-3aba367f798f-kube-api-access-82pjw\") pod \"authorino-66b54f7d67-rdqzr\" (UID: \"dc73575f-364a-4eb5-8f53-3aba367f798f\") " pod="kuadrant-system/authorino-66b54f7d67-rdqzr"
Sep 14 03:01:34.721928 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:01:34.721884    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dc73575f-364a-4eb5-8f53-3aba367f798f-tls-cert\") pod \"authorino-66b54f7d67-rdqzr\" (UID: \"dc73575f-364a-4eb5-8f53-3aba367f798f\") " pod="kuadrant-system/authorino-66b54f7d67-rdqzr"
Sep 14 03:01:34.722065 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:01:34.721949    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82pjw\" (UniqueName: \"kubernetes.io/projected/dc73575f-364a-4eb5-8f53-3aba367f798f-kube-api-access-82pjw\") pod \"authorino-66b54f7d67-rdqzr\" (UID: \"dc73575f-364a-4eb5-8f53-3aba367f798f\") " pod="kuadrant-system/authorino-66b54f7d67-rdqzr"
Sep 14 03:01:34.724438 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:01:34.724420    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dc73575f-364a-4eb5-8f53-3aba367f798f-tls-cert\") pod \"authorino-66b54f7d67-rdqzr\" (UID: \"dc73575f-364a-4eb5-8f53-3aba367f798f\") " pod="kuadrant-system/authorino-66b54f7d67-rdqzr"
Sep 14 03:01:34.731314 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:01:34.731295    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82pjw\" (UniqueName: \"kubernetes.io/projected/dc73575f-364a-4eb5-8f53-3aba367f798f-kube-api-access-82pjw\") pod \"authorino-66b54f7d67-rdqzr\" (UID: \"dc73575f-364a-4eb5-8f53-3aba367f798f\") " pod="kuadrant-system/authorino-66b54f7d67-rdqzr"
Sep 14 03:01:34.856094 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:01:34.856031    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66b54f7d67-rdqzr"
Sep 14 03:01:35.126365 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:01:35.126290    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66b54f7d67-rdqzr"]
Sep 14 03:01:35.128916 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:01:35.128879    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc73575f_364a_4eb5_8f53_3aba367f798f.slice/crio-d3ee04fe4befb9bb4b7ec82715696178c4231882a532491ac5ec806760ebede1 WatchSource:0}: Error finding container d3ee04fe4befb9bb4b7ec82715696178c4231882a532491ac5ec806760ebede1: Status 404 returned error can't find the container with id d3ee04fe4befb9bb4b7ec82715696178c4231882a532491ac5ec806760ebede1
Sep 14 03:01:36.097348 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:01:36.097307    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66b54f7d67-rdqzr" event={"ID":"dc73575f-364a-4eb5-8f53-3aba367f798f","Type":"ContainerStarted","Data":"d3ee04fe4befb9bb4b7ec82715696178c4231882a532491ac5ec806760ebede1"}
Sep 14 03:01:37.103711 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:01:37.103675    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66b54f7d67-rdqzr" event={"ID":"dc73575f-364a-4eb5-8f53-3aba367f798f","Type":"ContainerStarted","Data":"3140fb4f9086462376ae55dfac72287c7e92af5353e7aa0fc6e6345d8aa684c1"}
Sep 14 03:01:37.121029 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:01:37.120984    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-66b54f7d67-rdqzr" podStartSLOduration=1.702066273 podStartE2EDuration="3.120970546s" totalImagesPullingTime="1.418904273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:01:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:01:37.12063135 +0000 UTC m=+559.906693263" watchObservedRunningTime="2026-09-14 03:01:37.120970546 +0000 UTC m=+559.907032458"
Sep 14 03:02:33.958938 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:33.958909    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"]
Sep 14 03:02:33.963020 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:33.962997    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:33.966728 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:33.966699    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 14 03:02:33.966826 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:33.966776    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 14 03:02:33.966826 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:33.966705    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wdpm7\""
Sep 14 03:02:33.966826 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:33.966816    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 14 03:02:33.979152 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:33.979129    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"]
Sep 14 03:02:34.075500 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.075474    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db52f177-a321-4cb3-88d0-05a91f888574-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.075598 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.075503    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7njng\" (UniqueName: \"kubernetes.io/projected/db52f177-a321-4cb3-88d0-05a91f888574-kube-api-access-7njng\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.075598 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.075525    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db52f177-a321-4cb3-88d0-05a91f888574-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.075673 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.075630    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db52f177-a321-4cb3-88d0-05a91f888574-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.075673 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.075662    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db52f177-a321-4cb3-88d0-05a91f888574-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.075738 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.075686    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db52f177-a321-4cb3-88d0-05a91f888574-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.176270 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.176241    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db52f177-a321-4cb3-88d0-05a91f888574-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.176405 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.176275    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7njng\" (UniqueName: \"kubernetes.io/projected/db52f177-a321-4cb3-88d0-05a91f888574-kube-api-access-7njng\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.176405 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.176297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db52f177-a321-4cb3-88d0-05a91f888574-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.176405 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.176353    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db52f177-a321-4cb3-88d0-05a91f888574-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.176405 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.176385    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db52f177-a321-4cb3-88d0-05a91f888574-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.176601 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.176420    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db52f177-a321-4cb3-88d0-05a91f888574-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.176705 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.176680    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db52f177-a321-4cb3-88d0-05a91f888574-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.176769 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.176738    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db52f177-a321-4cb3-88d0-05a91f888574-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.176839 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.176818    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db52f177-a321-4cb3-88d0-05a91f888574-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.178691 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.178672    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db52f177-a321-4cb3-88d0-05a91f888574-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.178854 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.178835    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db52f177-a321-4cb3-88d0-05a91f888574-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.187279 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.187258    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7njng\" (UniqueName: \"kubernetes.io/projected/db52f177-a321-4cb3-88d0-05a91f888574-kube-api-access-7njng\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp\" (UID: \"db52f177-a321-4cb3-88d0-05a91f888574\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.272856 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.272830    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:34.552715 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.552598    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"]
Sep 14 03:02:34.555579 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:02:34.555553    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb52f177_a321_4cb3_88d0_05a91f888574.slice/crio-1e79b594b7d46fb803068b5c4806ac08af7e3933a4fe1706a489420bab5378bd WatchSource:0}: Error finding container 1e79b594b7d46fb803068b5c4806ac08af7e3933a4fe1706a489420bab5378bd: Status 404 returned error can't find the container with id 1e79b594b7d46fb803068b5c4806ac08af7e3933a4fe1706a489420bab5378bd
Sep 14 03:02:34.556914 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:34.556892    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 03:02:35.358065 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:35.358021    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp" event={"ID":"db52f177-a321-4cb3-88d0-05a91f888574","Type":"ContainerStarted","Data":"1e79b594b7d46fb803068b5c4806ac08af7e3933a4fe1706a489420bab5378bd"}
Sep 14 03:02:36.363854 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:36.363823    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp" event={"ID":"db52f177-a321-4cb3-88d0-05a91f888574","Type":"ContainerStarted","Data":"c087b8da6861943349c55f2355ef3918c74abc2b03bfe945dbe719e3ae7aeccd"}
Sep 14 03:02:36.382645 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:36.382598    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp" podStartSLOduration=2.3406940560000002 podStartE2EDuration="3.382582273s" totalImagesPullingTime="1.041888217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:02:36.381901183 +0000 UTC m=+619.167963109" watchObservedRunningTime="2026-09-14 03:02:36.382582273 +0000 UTC m=+619.168644186"
Sep 14 03:02:44.273493 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:44.273419    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:44.273493 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:44.273487    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:44.285920 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:44.285895    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:44.407763 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:44.407735    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp"
Sep 14 03:02:48.253612 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.253576    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"]
Sep 14 03:02:48.257281 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.257261    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.259890 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.259870    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 14 03:02:48.266386 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.266364    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"]
Sep 14 03:02:48.393115 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.393088    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed62ad4e-025d-4589-aca5-2eb249f62739-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.393293 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.393130    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed62ad4e-025d-4589-aca5-2eb249f62739-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.393293 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.393154    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed62ad4e-025d-4589-aca5-2eb249f62739-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.393293 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.393252    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed62ad4e-025d-4589-aca5-2eb249f62739-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.393293 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.393284    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed62ad4e-025d-4589-aca5-2eb249f62739-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.393431 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.393324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rmw5\" (UniqueName: \"kubernetes.io/projected/ed62ad4e-025d-4589-aca5-2eb249f62739-kube-api-access-5rmw5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.494462 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.494430    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed62ad4e-025d-4589-aca5-2eb249f62739-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.494462 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.494466    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed62ad4e-025d-4589-aca5-2eb249f62739-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.494679 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.494489    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed62ad4e-025d-4589-aca5-2eb249f62739-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.494679 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.494506    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed62ad4e-025d-4589-aca5-2eb249f62739-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.494679 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.494536    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rmw5\" (UniqueName: \"kubernetes.io/projected/ed62ad4e-025d-4589-aca5-2eb249f62739-kube-api-access-5rmw5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.494679 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.494579    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed62ad4e-025d-4589-aca5-2eb249f62739-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.494901 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.494862    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed62ad4e-025d-4589-aca5-2eb249f62739-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.494955 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.494898    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed62ad4e-025d-4589-aca5-2eb249f62739-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.494996 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.494974    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed62ad4e-025d-4589-aca5-2eb249f62739-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.496861 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.496842    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed62ad4e-025d-4589-aca5-2eb249f62739-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.497320 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.497302    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed62ad4e-025d-4589-aca5-2eb249f62739-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.505867 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.505816    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rmw5\" (UniqueName: \"kubernetes.io/projected/ed62ad4e-025d-4589-aca5-2eb249f62739-kube-api-access-5rmw5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zh8mk\" (UID: \"ed62ad4e-025d-4589-aca5-2eb249f62739\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.567585 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.567560    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:48.840616 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:48.840593    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"]
Sep 14 03:02:48.842785 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:02:48.842749    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded62ad4e_025d_4589_aca5_2eb249f62739.slice/crio-4c7d6f04959b0ff235447f1dc1a4bbf81492dd1f0f73ce4a057ed86bc4cdb10d WatchSource:0}: Error finding container 4c7d6f04959b0ff235447f1dc1a4bbf81492dd1f0f73ce4a057ed86bc4cdb10d: Status 404 returned error can't find the container with id 4c7d6f04959b0ff235447f1dc1a4bbf81492dd1f0f73ce4a057ed86bc4cdb10d
Sep 14 03:02:49.419630 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:49.419542    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk" event={"ID":"ed62ad4e-025d-4589-aca5-2eb249f62739","Type":"ContainerStarted","Data":"6462a6c2b280b1183c64d5e3b00ad89eb39f9e5f3d7b8fcd4d1bdbdf2e29059c"}
Sep 14 03:02:49.419630 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:49.419598    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk" event={"ID":"ed62ad4e-025d-4589-aca5-2eb249f62739","Type":"ContainerStarted","Data":"4c7d6f04959b0ff235447f1dc1a4bbf81492dd1f0f73ce4a057ed86bc4cdb10d"}
Sep 14 03:02:49.443248 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:49.443189    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk" podStartSLOduration=1.230856675 podStartE2EDuration="1.443148661s" totalImagesPullingTime="212.291986ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:02:49.44170013 +0000 UTC m=+632.227762042" watchObservedRunningTime="2026-09-14 03:02:49.443148661 +0000 UTC m=+632.229210576"
Sep 14 03:02:58.568068 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:58.568021    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:58.568544 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:58.568103    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:58.581132 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:58.581097    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:02:59.476447 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:02:59.476418    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zh8mk"
Sep 14 03:03:03.160811 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.160780    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"]
Sep 14 03:03:03.165379 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.165355    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.168524 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.168501    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 14 03:03:03.177614 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.177591    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"]
Sep 14 03:03:03.318795 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.318764    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f800c2dd-db74-4a77-98db-95de661306b7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.318976 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.318846    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f800c2dd-db74-4a77-98db-95de661306b7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.318976 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.318872    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f800c2dd-db74-4a77-98db-95de661306b7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.318976 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.318945    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f800c2dd-db74-4a77-98db-95de661306b7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.319117 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.318978    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-629xr\" (UniqueName: \"kubernetes.io/projected/f800c2dd-db74-4a77-98db-95de661306b7-kube-api-access-629xr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.319117 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.319003    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f800c2dd-db74-4a77-98db-95de661306b7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.419688 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.419611    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f800c2dd-db74-4a77-98db-95de661306b7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.419688 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.419655    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f800c2dd-db74-4a77-98db-95de661306b7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.419688 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.419672    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f800c2dd-db74-4a77-98db-95de661306b7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.419928 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.419721    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f800c2dd-db74-4a77-98db-95de661306b7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.419928 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.419743    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-629xr\" (UniqueName: \"kubernetes.io/projected/f800c2dd-db74-4a77-98db-95de661306b7-kube-api-access-629xr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.419928 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.419764    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f800c2dd-db74-4a77-98db-95de661306b7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.420097 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.420069    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f800c2dd-db74-4a77-98db-95de661306b7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.420203 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.420123    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f800c2dd-db74-4a77-98db-95de661306b7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.420203 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.420156    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f800c2dd-db74-4a77-98db-95de661306b7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.422241 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.422220    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f800c2dd-db74-4a77-98db-95de661306b7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.422506 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.422481    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f800c2dd-db74-4a77-98db-95de661306b7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.430802 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.430777    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-629xr\" (UniqueName: \"kubernetes.io/projected/f800c2dd-db74-4a77-98db-95de661306b7-kube-api-access-629xr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t92th\" (UID: \"f800c2dd-db74-4a77-98db-95de661306b7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.476742 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.476715    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:03.750994 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:03.750938    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"]
Sep 14 03:03:03.753046 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:03:03.753018    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf800c2dd_db74_4a77_98db_95de661306b7.slice/crio-48e2be65b4dabb4331dfedd71be7a229ca6569fa1a82358864fc354611f00ca8 WatchSource:0}: Error finding container 48e2be65b4dabb4331dfedd71be7a229ca6569fa1a82358864fc354611f00ca8: Status 404 returned error can't find the container with id 48e2be65b4dabb4331dfedd71be7a229ca6569fa1a82358864fc354611f00ca8
Sep 14 03:03:04.488917 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:04.488882    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th" event={"ID":"f800c2dd-db74-4a77-98db-95de661306b7","Type":"ContainerStarted","Data":"ab1bb51e311d053aeb09a0f9c8b7fa2082109ed82d4016722f95e8454737c5b8"}
Sep 14 03:03:04.489403 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:04.488944    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th" event={"ID":"f800c2dd-db74-4a77-98db-95de661306b7","Type":"ContainerStarted","Data":"48e2be65b4dabb4331dfedd71be7a229ca6569fa1a82358864fc354611f00ca8"}
Sep 14 03:03:04.512402 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:04.512360    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th" podStartSLOduration=1.347232159 podStartE2EDuration="1.512345372s" totalImagesPullingTime="165.113213ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:03:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:03:04.511687623 +0000 UTC m=+647.297749548" watchObservedRunningTime="2026-09-14 03:03:04.512345372 +0000 UTC m=+647.298407284"
Sep 14 03:03:13.477327 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:13.477231    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:13.477726 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:13.477369    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:13.490135 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:13.490113    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:03:13.540119 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:03:13.540093    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t92th"
Sep 14 03:04:43.655233 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:43.655195    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66b54f7d67-rdqzr"]
Sep 14 03:04:43.655851 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:43.655418    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-66b54f7d67-rdqzr" podUID="dc73575f-364a-4eb5-8f53-3aba367f798f" containerName="authorino" containerID="cri-o://3140fb4f9086462376ae55dfac72287c7e92af5353e7aa0fc6e6345d8aa684c1" gracePeriod=30
Sep 14 03:04:43.901849 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:43.901828    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66b54f7d67-rdqzr"
Sep 14 03:04:43.947944 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:43.947861    2579 generic.go:356] "Generic (PLEG): container finished" podID="dc73575f-364a-4eb5-8f53-3aba367f798f" containerID="3140fb4f9086462376ae55dfac72287c7e92af5353e7aa0fc6e6345d8aa684c1" exitCode=0
Sep 14 03:04:43.947944 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:43.947906    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66b54f7d67-rdqzr"
Sep 14 03:04:43.947944 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:43.947917    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66b54f7d67-rdqzr" event={"ID":"dc73575f-364a-4eb5-8f53-3aba367f798f","Type":"ContainerDied","Data":"3140fb4f9086462376ae55dfac72287c7e92af5353e7aa0fc6e6345d8aa684c1"}
Sep 14 03:04:43.948189 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:43.947959    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66b54f7d67-rdqzr" event={"ID":"dc73575f-364a-4eb5-8f53-3aba367f798f","Type":"ContainerDied","Data":"d3ee04fe4befb9bb4b7ec82715696178c4231882a532491ac5ec806760ebede1"}
Sep 14 03:04:43.948189 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:43.947980    2579 scope.go:117] "RemoveContainer" containerID="3140fb4f9086462376ae55dfac72287c7e92af5353e7aa0fc6e6345d8aa684c1"
Sep 14 03:04:43.956467 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:43.956450    2579 scope.go:117] "RemoveContainer" containerID="3140fb4f9086462376ae55dfac72287c7e92af5353e7aa0fc6e6345d8aa684c1"
Sep 14 03:04:43.956719 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:04:43.956703    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3140fb4f9086462376ae55dfac72287c7e92af5353e7aa0fc6e6345d8aa684c1\": container with ID starting with 3140fb4f9086462376ae55dfac72287c7e92af5353e7aa0fc6e6345d8aa684c1 not found: ID does not exist" containerID="3140fb4f9086462376ae55dfac72287c7e92af5353e7aa0fc6e6345d8aa684c1"
Sep 14 03:04:43.956773 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:43.956730    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3140fb4f9086462376ae55dfac72287c7e92af5353e7aa0fc6e6345d8aa684c1"} err="failed to get container status \"3140fb4f9086462376ae55dfac72287c7e92af5353e7aa0fc6e6345d8aa684c1\": rpc error: code = NotFound desc = could not find container \"3140fb4f9086462376ae55dfac72287c7e92af5353e7aa0fc6e6345d8aa684c1\": container with ID starting with 3140fb4f9086462376ae55dfac72287c7e92af5353e7aa0fc6e6345d8aa684c1 not found: ID does not exist"
Sep 14 03:04:44.029601 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:44.029573    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82pjw\" (UniqueName: \"kubernetes.io/projected/dc73575f-364a-4eb5-8f53-3aba367f798f-kube-api-access-82pjw\") pod \"dc73575f-364a-4eb5-8f53-3aba367f798f\" (UID: \"dc73575f-364a-4eb5-8f53-3aba367f798f\") "
Sep 14 03:04:44.029703 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:44.029608    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dc73575f-364a-4eb5-8f53-3aba367f798f-tls-cert\") pod \"dc73575f-364a-4eb5-8f53-3aba367f798f\" (UID: \"dc73575f-364a-4eb5-8f53-3aba367f798f\") "
Sep 14 03:04:44.031768 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:44.031740    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc73575f-364a-4eb5-8f53-3aba367f798f-kube-api-access-82pjw" (OuterVolumeSpecName: "kube-api-access-82pjw") pod "dc73575f-364a-4eb5-8f53-3aba367f798f" (UID: "dc73575f-364a-4eb5-8f53-3aba367f798f"). InnerVolumeSpecName "kube-api-access-82pjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:04:44.039757 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:44.039732    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc73575f-364a-4eb5-8f53-3aba367f798f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "dc73575f-364a-4eb5-8f53-3aba367f798f" (UID: "dc73575f-364a-4eb5-8f53-3aba367f798f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:04:44.131083 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:44.131055    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82pjw\" (UniqueName: \"kubernetes.io/projected/dc73575f-364a-4eb5-8f53-3aba367f798f-kube-api-access-82pjw\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:04:44.131083 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:44.131078    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dc73575f-364a-4eb5-8f53-3aba367f798f-tls-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:04:44.306778 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:44.306750    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66b54f7d67-rdqzr"]
Sep 14 03:04:44.309200 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:44.309163    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-66b54f7d67-rdqzr"]
Sep 14 03:04:45.743190 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:04:45.743153    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc73575f-364a-4eb5-8f53-3aba367f798f" path="/var/lib/kubelet/pods/dc73575f-364a-4eb5-8f53-3aba367f798f/volumes"
Sep 14 03:05:01.347771 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.347739    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"]
Sep 14 03:05:01.348159 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.348062    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc73575f-364a-4eb5-8f53-3aba367f798f" containerName="authorino"
Sep 14 03:05:01.348159 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.348073    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc73575f-364a-4eb5-8f53-3aba367f798f" containerName="authorino"
Sep 14 03:05:01.348159 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.348137    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc73575f-364a-4eb5-8f53-3aba367f798f" containerName="authorino"
Sep 14 03:05:01.352670 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.352651    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.373161 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.373139    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-3a752f-openshift-default-dockercfg-5c8qp\""
Sep 14 03:05:01.395121 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.395090    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"]
Sep 14 03:05:01.463901 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.463877    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istiod-ca-cert\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.464047 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.463916    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-envoy\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.464047 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.463940    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-data\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.464047 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.463961    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-workload-socket\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.464047 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.463976    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-token\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.464047 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.464021    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-podinfo\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.464047 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.464046    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wh2z\" (UniqueName: \"kubernetes.io/projected/f79d663e-c9e2-4040-8dfe-c21d73cd0198-kube-api-access-7wh2z\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.464281 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.464103    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-credential-socket\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.464281 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.464198    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-workload-certs\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.565051 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.565009    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-podinfo\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.565051 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.565046    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wh2z\" (UniqueName: \"kubernetes.io/projected/f79d663e-c9e2-4040-8dfe-c21d73cd0198-kube-api-access-7wh2z\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.565289 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.565208    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-credential-socket\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.565289 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.565277    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-workload-certs\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.565381 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.565336    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istiod-ca-cert\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.565435 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.565384    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-envoy\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.565435 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.565421    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-data\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.565543 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.565455    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-workload-socket\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.565543 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.565484    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-token\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.565733 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.565711    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-workload-certs\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.566069 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.566046    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istiod-ca-cert\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.566321 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.566046    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-data\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.566627 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.566083    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-workload-socket\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.566741 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.566124    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-credential-socket\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.568342 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.568310    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-envoy\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.568425 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.568385    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-podinfo\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.578850 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.578802    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wh2z\" (UniqueName: \"kubernetes.io/projected/f79d663e-c9e2-4040-8dfe-c21d73cd0198-kube-api-access-7wh2z\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.582984 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.582966    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-token\") pod \"e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") " pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.666401 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.666311    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:01.967451 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:01.967371    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"]
Sep 14 03:05:01.970488 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:05:01.970460    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf79d663e_c9e2_4040_8dfe_c21d73cd0198.slice/crio-3a55d35e50cd6d7d77a8c08ecb2f8c65672cb1f72bf2999a831869e2e44b2c4f WatchSource:0}: Error finding container 3a55d35e50cd6d7d77a8c08ecb2f8c65672cb1f72bf2999a831869e2e44b2c4f: Status 404 returned error can't find the container with id 3a55d35e50cd6d7d77a8c08ecb2f8c65672cb1f72bf2999a831869e2e44b2c4f
Sep 14 03:05:02.044039 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:02.044009    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk" event={"ID":"f79d663e-c9e2-4040-8dfe-c21d73cd0198","Type":"ContainerStarted","Data":"3a55d35e50cd6d7d77a8c08ecb2f8c65672cb1f72bf2999a831869e2e44b2c4f"}
Sep 14 03:05:04.830081 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:04.830046    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:04.830393 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:04.830114    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:04.830393 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:04.830141    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:05.064093 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:05.064055    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk" event={"ID":"f79d663e-c9e2-4040-8dfe-c21d73cd0198","Type":"ContainerStarted","Data":"7aee3806bdf0c82fd214dbf3cd012b0ebf9fecf3844e8005f86dd88fcc0e243a"}
Sep 14 03:05:05.666825 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:05.666793    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:05.671445 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:05.671422    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:05.731008 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:05.730961    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk" podStartSLOduration=1.8735767330000002 podStartE2EDuration="4.730945347s" totalImagesPullingTime="2.857368614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:05:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:05:05.108359458 +0000 UTC m=+767.894421371" watchObservedRunningTime="2026-09-14 03:05:05.730945347 +0000 UTC m=+768.517007259"
Sep 14 03:05:06.068316 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:06.068290    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:06.069317 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:06.069295    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:07.549052 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:07.549023    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"]
Sep 14 03:05:07.552471 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:07.552454    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"
Sep 14 03:05:07.555480 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:07.555458    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-9c9f1e-dockercfg-4czcr\""
Sep 14 03:05:07.555598 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:07.555529    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-9c9f1e\""
Sep 14 03:05:07.565702 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:07.565681    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"]
Sep 14 03:05:07.720038 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:07.720003    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d70a57b9-05c5-47ac-9ad7-691033cc0edf-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb\" (UID: \"d70a57b9-05c5-47ac-9ad7-691033cc0edf\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"
Sep 14 03:05:07.720220 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:07.720075    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8th7d\" (UniqueName: \"kubernetes.io/projected/d70a57b9-05c5-47ac-9ad7-691033cc0edf-kube-api-access-8th7d\") pod \"payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb\" (UID: \"d70a57b9-05c5-47ac-9ad7-691033cc0edf\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"
Sep 14 03:05:07.820581 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:07.820504    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8th7d\" (UniqueName: \"kubernetes.io/projected/d70a57b9-05c5-47ac-9ad7-691033cc0edf-kube-api-access-8th7d\") pod \"payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb\" (UID: \"d70a57b9-05c5-47ac-9ad7-691033cc0edf\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"
Sep 14 03:05:07.820581 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:07.820562    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d70a57b9-05c5-47ac-9ad7-691033cc0edf-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb\" (UID: \"d70a57b9-05c5-47ac-9ad7-691033cc0edf\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"
Sep 14 03:05:07.821227 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:07.821210    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d70a57b9-05c5-47ac-9ad7-691033cc0edf-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb\" (UID: \"d70a57b9-05c5-47ac-9ad7-691033cc0edf\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"
Sep 14 03:05:07.840982 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:07.840952    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8th7d\" (UniqueName: \"kubernetes.io/projected/d70a57b9-05c5-47ac-9ad7-691033cc0edf-kube-api-access-8th7d\") pod \"payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb\" (UID: \"d70a57b9-05c5-47ac-9ad7-691033cc0edf\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"
Sep 14 03:05:07.862400 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:07.862374    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"
Sep 14 03:05:08.149886 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:08.149806    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"]
Sep 14 03:05:08.153390 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:05:08.153361    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd70a57b9_05c5_47ac_9ad7_691033cc0edf.slice/crio-62669acc8def4c440299594e6bec094d618a1964eb9499bf12d38c3eb2c8672d WatchSource:0}: Error finding container 62669acc8def4c440299594e6bec094d618a1964eb9499bf12d38c3eb2c8672d: Status 404 returned error can't find the container with id 62669acc8def4c440299594e6bec094d618a1964eb9499bf12d38c3eb2c8672d
Sep 14 03:05:09.024830 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:09.024795    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"]
Sep 14 03:05:09.029307 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:09.029291    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"
Sep 14 03:05:09.032994 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:09.032959    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-f002a1-dockercfg-w2bxv\""
Sep 14 03:05:09.033124 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:09.033058    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-f002a1\""
Sep 14 03:05:09.049201 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:09.049164    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"]
Sep 14 03:05:09.083624 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:09.083541    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb" event={"ID":"d70a57b9-05c5-47ac-9ad7-691033cc0edf","Type":"ContainerStarted","Data":"62669acc8def4c440299594e6bec094d618a1964eb9499bf12d38c3eb2c8672d"}
Sep 14 03:05:09.131747 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:09.131718    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x8gb\" (UniqueName: \"kubernetes.io/projected/6712cbe0-242b-4630-b044-6daa82d59a6a-kube-api-access-9x8gb\") pod \"payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt\" (UID: \"6712cbe0-242b-4630-b044-6daa82d59a6a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"
Sep 14 03:05:09.131898 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:09.131763    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6712cbe0-242b-4630-b044-6daa82d59a6a-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt\" (UID: \"6712cbe0-242b-4630-b044-6daa82d59a6a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"
Sep 14 03:05:09.233107 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:09.233072    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6712cbe0-242b-4630-b044-6daa82d59a6a-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt\" (UID: \"6712cbe0-242b-4630-b044-6daa82d59a6a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"
Sep 14 03:05:09.233294 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:09.233252    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x8gb\" (UniqueName: \"kubernetes.io/projected/6712cbe0-242b-4630-b044-6daa82d59a6a-kube-api-access-9x8gb\") pod \"payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt\" (UID: \"6712cbe0-242b-4630-b044-6daa82d59a6a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"
Sep 14 03:05:09.234022 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:09.233998    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6712cbe0-242b-4630-b044-6daa82d59a6a-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt\" (UID: \"6712cbe0-242b-4630-b044-6daa82d59a6a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"
Sep 14 03:05:09.250316 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:09.250290    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x8gb\" (UniqueName: \"kubernetes.io/projected/6712cbe0-242b-4630-b044-6daa82d59a6a-kube-api-access-9x8gb\") pod \"payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt\" (UID: \"6712cbe0-242b-4630-b044-6daa82d59a6a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"
Sep 14 03:05:09.340041 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:09.339964    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"
Sep 14 03:05:09.628645 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:09.628612    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"]
Sep 14 03:05:09.884694 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:05:09.884623    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6712cbe0_242b_4630_b044_6daa82d59a6a.slice/crio-a1fa41c60baa043e69c60ae031edcee5450ac71b2ada4183c21d60ef5948dd63 WatchSource:0}: Error finding container a1fa41c60baa043e69c60ae031edcee5450ac71b2ada4183c21d60ef5948dd63: Status 404 returned error can't find the container with id a1fa41c60baa043e69c60ae031edcee5450ac71b2ada4183c21d60ef5948dd63
Sep 14 03:05:10.090671 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:10.090644    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb" event={"ID":"d70a57b9-05c5-47ac-9ad7-691033cc0edf","Type":"ContainerStarted","Data":"5c3972e0ba3410c26c96f4d1f0ec6a0667ad53206c0924492cb8cd550329adc4"}
Sep 14 03:05:10.091059 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:10.090695    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"
Sep 14 03:05:10.091986 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:10.091963    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt" event={"ID":"6712cbe0-242b-4630-b044-6daa82d59a6a","Type":"ContainerStarted","Data":"a1fa41c60baa043e69c60ae031edcee5450ac71b2ada4183c21d60ef5948dd63"}
Sep 14 03:05:10.147130 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:10.147050    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb" podStartSLOduration=1.364448264 podStartE2EDuration="3.147027132s" totalImagesPullingTime="1.782578868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:05:10.14582186 +0000 UTC m=+772.931883775" watchObservedRunningTime="2026-09-14 03:05:10.147027132 +0000 UTC m=+772.933089045"
Sep 14 03:05:11.100498 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:11.100461    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt" event={"ID":"6712cbe0-242b-4630-b044-6daa82d59a6a","Type":"ContainerStarted","Data":"27f1db36f1b396ff062ec3c9dcc9f32e7d148f3dae1b81cb3a32bb53b88d0dc6"}
Sep 14 03:05:11.100948 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:11.100642    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"
Sep 14 03:05:11.122516 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:11.122466    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt" podStartSLOduration=2.669606485 podStartE2EDuration="3.122452115s" totalImagesPullingTime="452.84563ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:05:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:05:11.121355709 +0000 UTC m=+773.907417623" watchObservedRunningTime="2026-09-14 03:05:11.122452115 +0000 UTC m=+773.908514027"
Sep 14 03:05:12.098477 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.098434    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"]
Sep 14 03:05:12.102000 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.101978    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.104718 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.104698    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1493ac87-openshift-default-dockercfg-7gfmh\""
Sep 14 03:05:12.117817 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.117789    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"]
Sep 14 03:05:12.262591 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.262556    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-data\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.262786 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.262624    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-envoy\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.262786 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.262648    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-podinfo\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.262786 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.262671    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69vlz\" (UniqueName: \"kubernetes.io/projected/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-kube-api-access-69vlz\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.262786 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.262698    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-workload-certs\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.262786 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.262728    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istiod-ca-cert\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.263050 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.262806    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-token\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.263050 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.262914    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-credential-socket\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.263050 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.262957    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-workload-socket\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.363420 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.363345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-token\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.363420 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.363403    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-credential-socket\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.363623 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.363429    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-workload-socket\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.363623 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.363473    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-data\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.363623 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.363560    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-envoy\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.363623 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.363586    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-podinfo\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.363623 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.363610    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69vlz\" (UniqueName: \"kubernetes.io/projected/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-kube-api-access-69vlz\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.363862 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.363635    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-workload-certs\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.363862 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.363661    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istiod-ca-cert\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.363961 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.363935    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-workload-socket\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.364194 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.364144    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-workload-certs\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.364299 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.364287    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-credential-socket\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.364476 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.364300    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-data\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.364476 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.364394    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istiod-ca-cert\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.366374 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.366351    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-podinfo\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.366556 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.366535    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-envoy\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.372595 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.372568    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-token\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.375463 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.375442    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69vlz\" (UniqueName: \"kubernetes.io/projected/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-kube-api-access-69vlz\") pod \"e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") " pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.413407 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.413383    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:12.759553 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.759526    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"]
Sep 14 03:05:12.760847 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:05:12.760821    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31ec3d82_9ba0_46d6_b5fc_c51f6c45e369.slice/crio-e4598d4450497d3e9134f9ec366fa6d04814eec3f6f44087875f68e0c6cb98a2 WatchSource:0}: Error finding container e4598d4450497d3e9134f9ec366fa6d04814eec3f6f44087875f68e0c6cb98a2: Status 404 returned error can't find the container with id e4598d4450497d3e9134f9ec366fa6d04814eec3f6f44087875f68e0c6cb98a2
Sep 14 03:05:12.764072 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.764041    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:12.764135 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.764108    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:12.764197 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:12.764135    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:13.113861 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:13.113783    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp" event={"ID":"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369","Type":"ContainerStarted","Data":"27ecac2f88dc31040538c95e88b0de35c5f98de27c5d6f29c0ccf5f423f1fdb7"}
Sep 14 03:05:13.113861 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:13.113834    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp" event={"ID":"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369","Type":"ContainerStarted","Data":"e4598d4450497d3e9134f9ec366fa6d04814eec3f6f44087875f68e0c6cb98a2"}
Sep 14 03:05:13.184411 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:13.184359    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp" podStartSLOduration=1.184342057 podStartE2EDuration="1.184342057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:05:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:05:13.1816875 +0000 UTC m=+775.967749412" watchObservedRunningTime="2026-09-14 03:05:13.184342057 +0000 UTC m=+775.970403966"
Sep 14 03:05:13.414450 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:13.414378    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:13.419295 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:13.419273    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:14.118736 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:14.118705    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:14.120294 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:14.120270    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:15.624425 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.624398    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"]
Sep 14 03:05:15.628223 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.628203    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:15.633165 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.633146    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 03:05:15.634327 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.634110    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:05:15.634327 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.634130    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-ab5750-serving-cert\""
Sep 14 03:05:15.634327 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.634212    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jf78h\""
Sep 14 03:05:15.634327 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.634130    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 03:05:15.647435 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.647414    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"]
Sep 14 03:05:15.687026 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.686999    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58162ed7-956f-4e12-9059-6d6139b1cc12-maas-api-tls\") pod \"maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:15.687139 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.687075    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58162ed7-956f-4e12-9059-6d6139b1cc12-metrics-certs\") pod \"maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:15.687242 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.687223    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcmc7\" (UniqueName: \"kubernetes.io/projected/58162ed7-956f-4e12-9059-6d6139b1cc12-kube-api-access-bcmc7\") pod \"maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:15.687301 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.687260    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58162ed7-956f-4e12-9059-6d6139b1cc12-openshift-service-ca\") pod \"maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:15.787864 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.787823    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58162ed7-956f-4e12-9059-6d6139b1cc12-metrics-certs\") pod \"maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:15.788055 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.787918    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcmc7\" (UniqueName: \"kubernetes.io/projected/58162ed7-956f-4e12-9059-6d6139b1cc12-kube-api-access-bcmc7\") pod \"maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:15.788055 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.787958    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58162ed7-956f-4e12-9059-6d6139b1cc12-openshift-service-ca\") pod \"maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:15.788055 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.787990    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58162ed7-956f-4e12-9059-6d6139b1cc12-maas-api-tls\") pod \"maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:15.788266 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:05:15.788076    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-serving-cert: secret "maas-api-e2e-autores-ab5750-serving-cert" not found
Sep 14 03:05:15.788266 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:05:15.788158    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58162ed7-956f-4e12-9059-6d6139b1cc12-maas-api-tls podName:58162ed7-956f-4e12-9059-6d6139b1cc12 nodeName:}" failed. No retries permitted until 2026-09-14 03:05:16.288134447 +0000 UTC m=+779.074196358 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/58162ed7-956f-4e12-9059-6d6139b1cc12-maas-api-tls") pod "maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w" (UID: "58162ed7-956f-4e12-9059-6d6139b1cc12") : secret "maas-api-e2e-autores-ab5750-serving-cert" not found
Sep 14 03:05:15.788863 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.788846    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58162ed7-956f-4e12-9059-6d6139b1cc12-openshift-service-ca\") pod \"maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:15.790467 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.790445    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58162ed7-956f-4e12-9059-6d6139b1cc12-metrics-certs\") pod \"maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:15.809796 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:15.809770    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcmc7\" (UniqueName: \"kubernetes.io/projected/58162ed7-956f-4e12-9059-6d6139b1cc12-kube-api-access-bcmc7\") pod \"maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:16.292369 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:16.292323    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58162ed7-956f-4e12-9059-6d6139b1cc12-maas-api-tls\") pod \"maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:16.294969 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:16.294938    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58162ed7-956f-4e12-9059-6d6139b1cc12-maas-api-tls\") pod \"maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:16.539130 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:16.539098    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:16.632406 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:16.632365    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"]
Sep 14 03:05:16.636120 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:16.636099    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"
Sep 14 03:05:16.638901 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:16.638875    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-ab5750\""
Sep 14 03:05:16.639216 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:16.639200    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-ab5750-dockercfg-25x8f\""
Sep 14 03:05:16.648603 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:16.648577    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"]
Sep 14 03:05:16.694293 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:16.694264    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2cwd\" (UniqueName: \"kubernetes.io/projected/0bbd8c1f-18eb-4dd3-be45-ebe4195a4668-kube-api-access-c2cwd\") pod \"payload-processing-e2e-autores-ab5750-6f8df94989-kx65f\" (UID: \"0bbd8c1f-18eb-4dd3-be45-ebe4195a4668\") " pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"
Sep 14 03:05:16.694408 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:16.694385    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bbd8c1f-18eb-4dd3-be45-ebe4195a4668-plugins-config-volume\") pod \"payload-processing-e2e-autores-ab5750-6f8df94989-kx65f\" (UID: \"0bbd8c1f-18eb-4dd3-be45-ebe4195a4668\") " pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"
Sep 14 03:05:16.795371 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:16.795284    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bbd8c1f-18eb-4dd3-be45-ebe4195a4668-plugins-config-volume\") pod \"payload-processing-e2e-autores-ab5750-6f8df94989-kx65f\" (UID: \"0bbd8c1f-18eb-4dd3-be45-ebe4195a4668\") " pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"
Sep 14 03:05:16.795371 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:16.795360    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2cwd\" (UniqueName: \"kubernetes.io/projected/0bbd8c1f-18eb-4dd3-be45-ebe4195a4668-kube-api-access-c2cwd\") pod \"payload-processing-e2e-autores-ab5750-6f8df94989-kx65f\" (UID: \"0bbd8c1f-18eb-4dd3-be45-ebe4195a4668\") " pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"
Sep 14 03:05:16.796140 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:16.796116    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bbd8c1f-18eb-4dd3-be45-ebe4195a4668-plugins-config-volume\") pod \"payload-processing-e2e-autores-ab5750-6f8df94989-kx65f\" (UID: \"0bbd8c1f-18eb-4dd3-be45-ebe4195a4668\") " pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"
Sep 14 03:05:16.806670 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:16.806646    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2cwd\" (UniqueName: \"kubernetes.io/projected/0bbd8c1f-18eb-4dd3-be45-ebe4195a4668-kube-api-access-c2cwd\") pod \"payload-processing-e2e-autores-ab5750-6f8df94989-kx65f\" (UID: \"0bbd8c1f-18eb-4dd3-be45-ebe4195a4668\") " pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"
Sep 14 03:05:16.816875 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:16.816857    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"]
Sep 14 03:05:16.819401 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:05:16.819375    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58162ed7_956f_4e12_9059_6d6139b1cc12.slice/crio-3a6b7aa05c1f4fd34395a1665e18d95cddf7b2dddda323e15d777f3dcb1e8bf2 WatchSource:0}: Error finding container 3a6b7aa05c1f4fd34395a1665e18d95cddf7b2dddda323e15d777f3dcb1e8bf2: Status 404 returned error can't find the container with id 3a6b7aa05c1f4fd34395a1665e18d95cddf7b2dddda323e15d777f3dcb1e8bf2
Sep 14 03:05:16.946848 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:16.946807    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"
Sep 14 03:05:17.134704 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:17.134624    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w" event={"ID":"58162ed7-956f-4e12-9059-6d6139b1cc12","Type":"ContainerStarted","Data":"3a6b7aa05c1f4fd34395a1665e18d95cddf7b2dddda323e15d777f3dcb1e8bf2"}
Sep 14 03:05:17.237054 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:17.236965    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"]
Sep 14 03:05:17.239985 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:05:17.239959    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bbd8c1f_18eb_4dd3_be45_ebe4195a4668.slice/crio-1090078eb6f5bc7dc64edef94402fa7b918cb3ed242011482ec99224f0f7efd4 WatchSource:0}: Error finding container 1090078eb6f5bc7dc64edef94402fa7b918cb3ed242011482ec99224f0f7efd4: Status 404 returned error can't find the container with id 1090078eb6f5bc7dc64edef94402fa7b918cb3ed242011482ec99224f0f7efd4
Sep 14 03:05:18.142984 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:18.142891    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f" event={"ID":"0bbd8c1f-18eb-4dd3-be45-ebe4195a4668","Type":"ContainerStarted","Data":"ec4d03d4e973d5c97011c21d9085d33c0ba3b7f9d037f42d57fe1dcdd19fb177"}
Sep 14 03:05:18.142984 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:18.142951    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f" event={"ID":"0bbd8c1f-18eb-4dd3-be45-ebe4195a4668","Type":"ContainerStarted","Data":"1090078eb6f5bc7dc64edef94402fa7b918cb3ed242011482ec99224f0f7efd4"}
Sep 14 03:05:18.143455 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:18.143041    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"
Sep 14 03:05:18.163058 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:18.162999    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f" podStartSLOduration=2.162979472 podStartE2EDuration="2.162979472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:05:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:05:18.16147368 +0000 UTC m=+780.947535594" watchObservedRunningTime="2026-09-14 03:05:18.162979472 +0000 UTC m=+780.949041388"
Sep 14 03:05:20.156110 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:20.156072    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w" event={"ID":"58162ed7-956f-4e12-9059-6d6139b1cc12","Type":"ContainerStarted","Data":"40cd3af0fab2f541ef9ba0e84030f8ee4bd68b83c47571ab30f43f55ff5c2e39"}
Sep 14 03:05:20.156599 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:20.156233    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:20.197093 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:20.197036    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w" podStartSLOduration=2.781291799 podStartE2EDuration="5.197017501s" totalImagesPullingTime="2.415725702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:05:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:05:20.194553219 +0000 UTC m=+782.980615134" watchObservedRunningTime="2026-09-14 03:05:20.197017501 +0000 UTC m=+782.983079415"
Sep 14 03:05:21.102973 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:21.102944    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"
Sep 14 03:05:22.109322 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:22.109297    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"
Sep 14 03:05:25.305078 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.305046    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"]
Sep 14 03:05:25.313923 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.313899    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.324662 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.324635    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"]
Sep 14 03:05:25.325126 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.325018    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-4eb26b-openshift-default-dockercfg-t9bxg\""
Sep 14 03:05:25.372692 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.372657    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f96c6625-2303-4192-91b5-69a588a947a0-istio-token\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.372820 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.372701    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f96c6625-2303-4192-91b5-69a588a947a0-istio-podinfo\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.372820 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.372772    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx77f\" (UniqueName: \"kubernetes.io/projected/f96c6625-2303-4192-91b5-69a588a947a0-kube-api-access-cx77f\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.372890 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.372821    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-credential-socket\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.372890 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.372860    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-workload-certs\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.372959 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.372943    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-istio-envoy\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.372992 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.372972    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-workload-socket\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.373025 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.372998    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-istio-data\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.373025 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.373013    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f96c6625-2303-4192-91b5-69a588a947a0-istiod-ca-cert\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.473701 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.473664    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-istio-data\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.473701 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.473699    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f96c6625-2303-4192-91b5-69a588a947a0-istiod-ca-cert\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.473930 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.473747    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f96c6625-2303-4192-91b5-69a588a947a0-istio-token\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.473930 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.473773    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f96c6625-2303-4192-91b5-69a588a947a0-istio-podinfo\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.473930 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.473820    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx77f\" (UniqueName: \"kubernetes.io/projected/f96c6625-2303-4192-91b5-69a588a947a0-kube-api-access-cx77f\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.473930 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.473853    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-credential-socket\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.473930 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.473906    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-workload-certs\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.474168 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.473969    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-istio-envoy\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.474168 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.473995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-workload-socket\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.474168 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.474126    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-istio-data\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.474359 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.474308    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-workload-socket\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.474415 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.474385    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-workload-certs\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.474524 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.474505    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-credential-socket\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.474572 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.474530    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f96c6625-2303-4192-91b5-69a588a947a0-istiod-ca-cert\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.476428 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.476396    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-istio-envoy\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.476594 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.476576    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f96c6625-2303-4192-91b5-69a588a947a0-istio-podinfo\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.482702 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.482678    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx77f\" (UniqueName: \"kubernetes.io/projected/f96c6625-2303-4192-91b5-69a588a947a0-kube-api-access-cx77f\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.482862 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.482845    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f96c6625-2303-4192-91b5-69a588a947a0-istio-token\") pod \"e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") " pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.644059 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.643979    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:25.947875 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.947841    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"]
Sep 14 03:05:25.948870 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:05:25.948846    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf96c6625_2303_4192_91b5_69a588a947a0.slice/crio-280f638359fb5bc6f6557ea86e9477f8b6607671aee36b7349e20f84913e90c5 WatchSource:0}: Error finding container 280f638359fb5bc6f6557ea86e9477f8b6607671aee36b7349e20f84913e90c5: Status 404 returned error can't find the container with id 280f638359fb5bc6f6557ea86e9477f8b6607671aee36b7349e20f84913e90c5
Sep 14 03:05:25.951724 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.951688    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:25.951816 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.951768    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:25.951877 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:25.951813    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:26.166559 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:26.166527    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:05:26.189193 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:26.189152    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj" event={"ID":"f96c6625-2303-4192-91b5-69a588a947a0","Type":"ContainerStarted","Data":"4b2e5a55901021eff3664a69293725d5f2df4bfc16ea7e945c2dc46b4c693863"}
Sep 14 03:05:26.189348 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:26.189226    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj" event={"ID":"f96c6625-2303-4192-91b5-69a588a947a0","Type":"ContainerStarted","Data":"280f638359fb5bc6f6557ea86e9477f8b6607671aee36b7349e20f84913e90c5"}
Sep 14 03:05:26.217246 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:26.216942    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj" podStartSLOduration=1.216907478 podStartE2EDuration="1.216907478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:05:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:05:26.215817753 +0000 UTC m=+789.001879667" watchObservedRunningTime="2026-09-14 03:05:26.216907478 +0000 UTC m=+789.002969391"
Sep 14 03:05:26.644897 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:26.644853    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:27.649978 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:27.649948    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:28.204489 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:28.204462    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:28.211996 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:28.211971    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:29.150798 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:29.150768    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"
Sep 14 03:05:33.256592 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:33.256561    2579 generic.go:356] "Generic (PLEG): container finished" podID="f79d663e-c9e2-4040-8dfe-c21d73cd0198" containerID="7aee3806bdf0c82fd214dbf3cd012b0ebf9fecf3844e8005f86dd88fcc0e243a" exitCode=0
Sep 14 03:05:33.257090 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:33.256630    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk" event={"ID":"f79d663e-c9e2-4040-8dfe-c21d73cd0198","Type":"ContainerDied","Data":"7aee3806bdf0c82fd214dbf3cd012b0ebf9fecf3844e8005f86dd88fcc0e243a"}
Sep 14 03:05:33.257162 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:33.257108    2579 scope.go:117] "RemoveContainer" containerID="7aee3806bdf0c82fd214dbf3cd012b0ebf9fecf3844e8005f86dd88fcc0e243a"
Sep 14 03:05:33.258301 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:33.258276    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:33.258379 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:33.258340    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:33.258379 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:33.258371    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:34.263992 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:34.263950    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk" event={"ID":"f79d663e-c9e2-4040-8dfe-c21d73cd0198","Type":"ContainerStarted","Data":"c12c2e14f847912f41d92af17c0617630dd200960e9e673d91ee7538a4b597b9"}
Sep 14 03:05:34.666795 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:34.666714    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:34.667544 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:34.667522    2579 patch_prober.go:28] interesting pod/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 14 03:05:34.667602 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:34.667567    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk" podUID="f79d663e-c9e2-4040-8dfe-c21d73cd0198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 14 03:05:35.015538 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.015497    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"]
Sep 14 03:05:35.022334 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.021418    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"
Sep 14 03:05:35.026847 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.026810    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-4eb26b\""
Sep 14 03:05:35.034393 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.033584    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-4eb26b-dockercfg-47jzl\""
Sep 14 03:05:35.064806 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.064778    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"]
Sep 14 03:05:35.079977 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.079940    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gnlq\" (UniqueName: \"kubernetes.io/projected/e5ce29f4-0f10-47d9-93ef-b306283df152-kube-api-access-5gnlq\") pod \"payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx\" (UID: \"e5ce29f4-0f10-47d9-93ef-b306283df152\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"
Sep 14 03:05:35.080141 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.080001    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5ce29f4-0f10-47d9-93ef-b306283df152-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx\" (UID: \"e5ce29f4-0f10-47d9-93ef-b306283df152\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"
Sep 14 03:05:35.110469 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.108465    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"]
Sep 14 03:05:35.116316 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.116265    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"
Sep 14 03:05:35.141555 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.141471    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"]
Sep 14 03:05:35.180741 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.180713    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6776e241-6255-4ddf-aa9a-0fdb133a5421-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l\" (UID: \"6776e241-6255-4ddf-aa9a-0fdb133a5421\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"
Sep 14 03:05:35.180905 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.180778    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gnlq\" (UniqueName: \"kubernetes.io/projected/e5ce29f4-0f10-47d9-93ef-b306283df152-kube-api-access-5gnlq\") pod \"payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx\" (UID: \"e5ce29f4-0f10-47d9-93ef-b306283df152\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"
Sep 14 03:05:35.180905 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.180808    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5ce29f4-0f10-47d9-93ef-b306283df152-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx\" (UID: \"e5ce29f4-0f10-47d9-93ef-b306283df152\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"
Sep 14 03:05:35.180905 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.180831    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2x8c\" (UniqueName: \"kubernetes.io/projected/6776e241-6255-4ddf-aa9a-0fdb133a5421-kube-api-access-w2x8c\") pod \"payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l\" (UID: \"6776e241-6255-4ddf-aa9a-0fdb133a5421\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"
Sep 14 03:05:35.182045 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.182021    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5ce29f4-0f10-47d9-93ef-b306283df152-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx\" (UID: \"e5ce29f4-0f10-47d9-93ef-b306283df152\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"
Sep 14 03:05:35.194318 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.193756    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gnlq\" (UniqueName: \"kubernetes.io/projected/e5ce29f4-0f10-47d9-93ef-b306283df152-kube-api-access-5gnlq\") pod \"payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx\" (UID: \"e5ce29f4-0f10-47d9-93ef-b306283df152\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"
Sep 14 03:05:35.281868 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.281793    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2x8c\" (UniqueName: \"kubernetes.io/projected/6776e241-6255-4ddf-aa9a-0fdb133a5421-kube-api-access-w2x8c\") pod \"payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l\" (UID: \"6776e241-6255-4ddf-aa9a-0fdb133a5421\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"
Sep 14 03:05:35.282241 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.281892    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6776e241-6255-4ddf-aa9a-0fdb133a5421-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l\" (UID: \"6776e241-6255-4ddf-aa9a-0fdb133a5421\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"
Sep 14 03:05:35.282759 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.282737    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6776e241-6255-4ddf-aa9a-0fdb133a5421-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l\" (UID: \"6776e241-6255-4ddf-aa9a-0fdb133a5421\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"
Sep 14 03:05:35.311387 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.311362    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2x8c\" (UniqueName: \"kubernetes.io/projected/6776e241-6255-4ddf-aa9a-0fdb133a5421-kube-api-access-w2x8c\") pod \"payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l\" (UID: \"6776e241-6255-4ddf-aa9a-0fdb133a5421\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"
Sep 14 03:05:35.343439 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.343322    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"
Sep 14 03:05:35.441581 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.441544    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"
Sep 14 03:05:35.667312 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.667278    2579 patch_prober.go:28] interesting pod/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 14 03:05:35.667467 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.667326    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk" podUID="f79d663e-c9e2-4040-8dfe-c21d73cd0198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 14 03:05:35.704158 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.704128    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"]
Sep 14 03:05:35.819266 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:35.817699    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"]
Sep 14 03:05:36.069389 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:36.069291    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:36.276028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:36.275995    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx" event={"ID":"e5ce29f4-0f10-47d9-93ef-b306283df152","Type":"ContainerStarted","Data":"b0a89ff085c36f13e7b93a7911ad1f44268b552e772425e56a77975290f061f6"}
Sep 14 03:05:36.276188 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:36.276045    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx" event={"ID":"e5ce29f4-0f10-47d9-93ef-b306283df152","Type":"ContainerStarted","Data":"1aec14b23a4017227a59ddecd798e61991dcacf23801b857b97bb45943fee6a4"}
Sep 14 03:05:36.276642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:36.276626    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"
Sep 14 03:05:36.278328 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:36.278241    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l" event={"ID":"6776e241-6255-4ddf-aa9a-0fdb133a5421","Type":"ContainerStarted","Data":"3748ef4c64e2ee34051738ae2d04b3efbcb617276f94763145f80f60f24cc5b9"}
Sep 14 03:05:36.278328 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:36.278283    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l" event={"ID":"6776e241-6255-4ddf-aa9a-0fdb133a5421","Type":"ContainerStarted","Data":"5abff48cb62c3763a4f3affc8eeac6206eafe62109e1f48654ae99c077ba6c54"}
Sep 14 03:05:36.278530 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:36.278386    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"
Sep 14 03:05:36.323081 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:36.322970    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx" podStartSLOduration=2.3229482839999998 podStartE2EDuration="2.322948284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:05:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:05:36.299362344 +0000 UTC m=+799.085424260" watchObservedRunningTime="2026-09-14 03:05:36.322948284 +0000 UTC m=+799.109010203"
Sep 14 03:05:36.666944 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:36.666858    2579 patch_prober.go:28] interesting pod/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 14 03:05:36.666944 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:36.666920    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk" podUID="f79d663e-c9e2-4040-8dfe-c21d73cd0198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 14 03:05:37.666746 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:37.666715    2579 patch_prober.go:28] interesting pod/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 14 03:05:37.667120 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:37.666768    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk" podUID="f79d663e-c9e2-4040-8dfe-c21d73cd0198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 14 03:05:38.666935 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:38.666851    2579 patch_prober.go:28] interesting pod/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 14 03:05:38.666935 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:38.666912    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk" podUID="f79d663e-c9e2-4040-8dfe-c21d73cd0198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 14 03:05:39.732241 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:39.730542    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:39.736001 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:39.735974    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:05:39.757135 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:39.757085    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l" podStartSLOduration=4.757069893 podStartE2EDuration="4.757069893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:05:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:05:36.32408783 +0000 UTC m=+799.110149744" watchObservedRunningTime="2026-09-14 03:05:39.757069893 +0000 UTC m=+802.543131806"
Sep 14 03:05:43.323489 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:43.323398    2579 generic.go:356] "Generic (PLEG): container finished" podID="31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" containerID="27ecac2f88dc31040538c95e88b0de35c5f98de27c5d6f29c0ccf5f423f1fdb7" exitCode=0
Sep 14 03:05:43.323489 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:43.323466    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp" event={"ID":"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369","Type":"ContainerDied","Data":"27ecac2f88dc31040538c95e88b0de35c5f98de27c5d6f29c0ccf5f423f1fdb7"}
Sep 14 03:05:43.323946 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:43.323873    2579 scope.go:117] "RemoveContainer" containerID="27ecac2f88dc31040538c95e88b0de35c5f98de27c5d6f29c0ccf5f423f1fdb7"
Sep 14 03:05:43.325072 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:43.325047    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:43.325130 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:43.325117    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:43.325200 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:43.325150    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:44.118643 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:44.118611    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:44.336015 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:44.335969    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp" event={"ID":"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369","Type":"ContainerStarted","Data":"8fd9905471a8056b298e8cf29ac62d60664f8489d7768e08f1b1a290f9a2629f"}
Sep 14 03:05:44.416080 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:44.415976    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:44.416279 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:44.416115    2579 patch_prober.go:28] interesting pod/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 14 03:05:44.416279 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:44.416166    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp" podUID="31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 14 03:05:45.414335 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:45.413860    2579 patch_prober.go:28] interesting pod/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 14 03:05:45.414335 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:45.413936    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp" podUID="31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 14 03:05:46.414052 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:46.414006    2579 patch_prober.go:28] interesting pod/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 14 03:05:46.414382 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:46.414077    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp" podUID="31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 14 03:05:47.284871 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:47.284843    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"
Sep 14 03:05:47.417568 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:47.417535    2579 patch_prober.go:28] interesting pod/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 14 03:05:47.418116 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:47.418088    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp" podUID="31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 14 03:05:48.296430 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:48.296392    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"
Sep 14 03:05:48.416200 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:48.415647    2579 patch_prober.go:28] interesting pod/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 14 03:05:48.416200 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:48.415729    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp" podUID="31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 14 03:05:49.034313 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:49.031756    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"]
Sep 14 03:05:49.034313 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:49.031978    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp" podUID="31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" containerName="istio-proxy" containerID="cri-o://8fd9905471a8056b298e8cf29ac62d60664f8489d7768e08f1b1a290f9a2629f" gracePeriod=30
Sep 14 03:05:52.388192 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:52.388089    2579 generic.go:356] "Generic (PLEG): container finished" podID="f96c6625-2303-4192-91b5-69a588a947a0" containerID="4b2e5a55901021eff3664a69293725d5f2df4bfc16ea7e945c2dc46b4c693863" exitCode=0
Sep 14 03:05:52.388192 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:52.388146    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj" event={"ID":"f96c6625-2303-4192-91b5-69a588a947a0","Type":"ContainerDied","Data":"4b2e5a55901021eff3664a69293725d5f2df4bfc16ea7e945c2dc46b4c693863"}
Sep 14 03:05:52.388681 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:52.388527    2579 scope.go:117] "RemoveContainer" containerID="4b2e5a55901021eff3664a69293725d5f2df4bfc16ea7e945c2dc46b4c693863"
Sep 14 03:05:52.389638 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:52.389607    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:52.389727 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:52.389675    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:52.389727 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:52.389704    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:05:53.397693 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:53.397638    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj" event={"ID":"f96c6625-2303-4192-91b5-69a588a947a0","Type":"ContainerStarted","Data":"423416f2a73e8a872615a8a30e2e4c60782c6e6be35054957bcca142b24b76fb"}
Sep 14 03:05:53.644576 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:53.644537    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:53.645764 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:53.645741    2579 patch_prober.go:28] interesting pod/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 14 03:05:53.645874 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:53.645795    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj" podUID="f96c6625-2303-4192-91b5-69a588a947a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 14 03:05:54.645252 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.645226    2579 patch_prober.go:28] interesting pod/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 14 03:05:54.645558 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.645278    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj" podUID="f96c6625-2303-4192-91b5-69a588a947a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 14 03:05:54.831491 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.831465    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:54.926009 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.925919    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-data\") pod \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") "
Sep 14 03:05:54.926009 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.925976    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69vlz\" (UniqueName: \"kubernetes.io/projected/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-kube-api-access-69vlz\") pod \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") "
Sep 14 03:05:54.926009 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.925993    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-token\") pod \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") "
Sep 14 03:05:54.926306 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.926033    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-credential-socket\") pod \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") "
Sep 14 03:05:54.926306 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.926049    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-workload-socket\") pod \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") "
Sep 14 03:05:54.926306 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.926078    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-workload-certs\") pod \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") "
Sep 14 03:05:54.926306 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.926104    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istiod-ca-cert\") pod \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") "
Sep 14 03:05:54.926306 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.926149    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-podinfo\") pod \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") "
Sep 14 03:05:54.926306 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.926204    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-envoy\") pod \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\" (UID: \"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369\") "
Sep 14 03:05:54.926306 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.926248    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-data" (OuterVolumeSpecName: "istio-data") pod "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" (UID: "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:05:54.926658 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.926494    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:05:54.926658 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.926592    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" (UID: "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:05:54.926658 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.926606    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" (UID: "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:05:54.926658 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.926624    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" (UID: "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:05:54.926877 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.926673    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" (UID: "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:05:54.928564 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.928533    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-token" (OuterVolumeSpecName: "istio-token") pod "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" (UID: "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:05:54.928668 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.928598    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" (UID: "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:05:54.928668 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.928604    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-kube-api-access-69vlz" (OuterVolumeSpecName: "kube-api-access-69vlz") pod "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" (UID: "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369"). InnerVolumeSpecName "kube-api-access-69vlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:05:54.928929 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:54.928914    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" (UID: "31ec3d82-9ba0-46d6-b5fc-c51f6c45e369"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:05:55.027932 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.027900    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69vlz\" (UniqueName: \"kubernetes.io/projected/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-kube-api-access-69vlz\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:05:55.027932 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.027936    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:05:55.028127 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.027948    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:05:55.028127 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.027956    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:05:55.028127 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.027964    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:05:55.028127 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.027973    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:05:55.028127 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.027980    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:05:55.028127 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.027987    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:05:55.368460 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.368430    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"]
Sep 14 03:05:55.368863 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.368848    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" containerName="istio-proxy"
Sep 14 03:05:55.368913 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.368866    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" containerName="istio-proxy"
Sep 14 03:05:55.368913 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.368877    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" containerName="istio-proxy"
Sep 14 03:05:55.368913 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.368882    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" containerName="istio-proxy"
Sep 14 03:05:55.369008 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.368976    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" containerName="istio-proxy"
Sep 14 03:05:55.369008 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.368988    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" containerName="istio-proxy"
Sep 14 03:05:55.375263 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.374167    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"
Sep 14 03:05:55.378981 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.378955    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-d8a641aa-dockercfg-w82zw\""
Sep 14 03:05:55.379095 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.378955    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-d8a641aa\""
Sep 14 03:05:55.404747 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.404701    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"]
Sep 14 03:05:55.428013 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.427977    2579 generic.go:356] "Generic (PLEG): container finished" podID="31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" containerID="8fd9905471a8056b298e8cf29ac62d60664f8489d7768e08f1b1a290f9a2629f" exitCode=0
Sep 14 03:05:55.428234 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.428089    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp" event={"ID":"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369","Type":"ContainerDied","Data":"8fd9905471a8056b298e8cf29ac62d60664f8489d7768e08f1b1a290f9a2629f"}
Sep 14 03:05:55.428234 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.428142    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp" event={"ID":"31ec3d82-9ba0-46d6-b5fc-c51f6c45e369","Type":"ContainerDied","Data":"e4598d4450497d3e9134f9ec366fa6d04814eec3f6f44087875f68e0c6cb98a2"}
Sep 14 03:05:55.428234 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.428200    2579 scope.go:117] "RemoveContainer" containerID="8fd9905471a8056b298e8cf29ac62d60664f8489d7768e08f1b1a290f9a2629f"
Sep 14 03:05:55.428403 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.428374    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"
Sep 14 03:05:55.431503 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.431473    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zwl4\" (UniqueName: \"kubernetes.io/projected/277f486c-44dd-4e5c-b285-64e8ca2f15a6-kube-api-access-2zwl4\") pod \"payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q\" (UID: \"277f486c-44dd-4e5c-b285-64e8ca2f15a6\") " pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"
Sep 14 03:05:55.431642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.431573    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/277f486c-44dd-4e5c-b285-64e8ca2f15a6-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q\" (UID: \"277f486c-44dd-4e5c-b285-64e8ca2f15a6\") " pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"
Sep 14 03:05:55.439812 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.439796    2579 scope.go:117] "RemoveContainer" containerID="27ecac2f88dc31040538c95e88b0de35c5f98de27c5d6f29c0ccf5f423f1fdb7"
Sep 14 03:05:55.450157 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.450065    2579 scope.go:117] "RemoveContainer" containerID="8fd9905471a8056b298e8cf29ac62d60664f8489d7768e08f1b1a290f9a2629f"
Sep 14 03:05:55.454646 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:05:55.454605    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fd9905471a8056b298e8cf29ac62d60664f8489d7768e08f1b1a290f9a2629f\": container with ID starting with 8fd9905471a8056b298e8cf29ac62d60664f8489d7768e08f1b1a290f9a2629f not found: ID does not exist" containerID="8fd9905471a8056b298e8cf29ac62d60664f8489d7768e08f1b1a290f9a2629f"
Sep 14 03:05:55.454768 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.454646    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fd9905471a8056b298e8cf29ac62d60664f8489d7768e08f1b1a290f9a2629f"} err="failed to get container status \"8fd9905471a8056b298e8cf29ac62d60664f8489d7768e08f1b1a290f9a2629f\": rpc error: code = NotFound desc = could not find container \"8fd9905471a8056b298e8cf29ac62d60664f8489d7768e08f1b1a290f9a2629f\": container with ID starting with 8fd9905471a8056b298e8cf29ac62d60664f8489d7768e08f1b1a290f9a2629f not found: ID does not exist"
Sep 14 03:05:55.454768 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.454671    2579 scope.go:117] "RemoveContainer" containerID="27ecac2f88dc31040538c95e88b0de35c5f98de27c5d6f29c0ccf5f423f1fdb7"
Sep 14 03:05:55.455167 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:05:55.455143    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27ecac2f88dc31040538c95e88b0de35c5f98de27c5d6f29c0ccf5f423f1fdb7\": container with ID starting with 27ecac2f88dc31040538c95e88b0de35c5f98de27c5d6f29c0ccf5f423f1fdb7 not found: ID does not exist" containerID="27ecac2f88dc31040538c95e88b0de35c5f98de27c5d6f29c0ccf5f423f1fdb7"
Sep 14 03:05:55.455355 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.455325    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27ecac2f88dc31040538c95e88b0de35c5f98de27c5d6f29c0ccf5f423f1fdb7"} err="failed to get container status \"27ecac2f88dc31040538c95e88b0de35c5f98de27c5d6f29c0ccf5f423f1fdb7\": rpc error: code = NotFound desc = could not find container \"27ecac2f88dc31040538c95e88b0de35c5f98de27c5d6f29c0ccf5f423f1fdb7\": container with ID starting with 27ecac2f88dc31040538c95e88b0de35c5f98de27c5d6f29c0ccf5f423f1fdb7 not found: ID does not exist"
Sep 14 03:05:55.456707 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.456677    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"]
Sep 14 03:05:55.458625 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.458605    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1493ac87-openshift-default-864ff78f7b-vrldp"]
Sep 14 03:05:55.533034 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.532991    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zwl4\" (UniqueName: \"kubernetes.io/projected/277f486c-44dd-4e5c-b285-64e8ca2f15a6-kube-api-access-2zwl4\") pod \"payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q\" (UID: \"277f486c-44dd-4e5c-b285-64e8ca2f15a6\") " pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"
Sep 14 03:05:55.533034 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.533037    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/277f486c-44dd-4e5c-b285-64e8ca2f15a6-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q\" (UID: \"277f486c-44dd-4e5c-b285-64e8ca2f15a6\") " pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"
Sep 14 03:05:55.533774 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.533752    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/277f486c-44dd-4e5c-b285-64e8ca2f15a6-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q\" (UID: \"277f486c-44dd-4e5c-b285-64e8ca2f15a6\") " pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"
Sep 14 03:05:55.546286 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.546265    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zwl4\" (UniqueName: \"kubernetes.io/projected/277f486c-44dd-4e5c-b285-64e8ca2f15a6-kube-api-access-2zwl4\") pod \"payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q\" (UID: \"277f486c-44dd-4e5c-b285-64e8ca2f15a6\") " pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"
Sep 14 03:05:55.644977 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.644908    2579 patch_prober.go:28] interesting pod/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 14 03:05:55.644977 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.644967    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj" podUID="f96c6625-2303-4192-91b5-69a588a947a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 14 03:05:55.685923 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.685885    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"
Sep 14 03:05:55.745767 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.745348    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31ec3d82-9ba0-46d6-b5fc-c51f6c45e369" path="/var/lib/kubelet/pods/31ec3d82-9ba0-46d6-b5fc-c51f6c45e369/volumes"
Sep 14 03:05:55.991275 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:55.991244    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"]
Sep 14 03:05:56.434979 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:56.434941    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q" event={"ID":"277f486c-44dd-4e5c-b285-64e8ca2f15a6","Type":"ContainerStarted","Data":"4e4c6d45773e1cf84472e2bec57edc9857320d78d833d150e0d3cf10b3ff8443"}
Sep 14 03:05:56.435200 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:56.435004    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q" event={"ID":"277f486c-44dd-4e5c-b285-64e8ca2f15a6","Type":"ContainerStarted","Data":"1db9c06bbbb9eb46befbf2f4330e1cc751c98102758e099bd06ce119e1b9610c"}
Sep 14 03:05:56.435200 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:56.435037    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"
Sep 14 03:05:56.453880 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:56.453828    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q" podStartSLOduration=1.453810332 podStartE2EDuration="1.453810332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:05:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:05:56.453505238 +0000 UTC m=+819.239567152" watchObservedRunningTime="2026-09-14 03:05:56.453810332 +0000 UTC m=+819.239872245"
Sep 14 03:05:56.644494 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:56.644463    2579 patch_prober.go:28] interesting pod/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 14 03:05:56.644662 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:56.644519    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj" podUID="f96c6625-2303-4192-91b5-69a588a947a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 14 03:05:57.644649 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:57.644613    2579 patch_prober.go:28] interesting pod/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 14 03:05:57.645106 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:57.644673    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj" podUID="f96c6625-2303-4192-91b5-69a588a947a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 14 03:05:58.205384 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:58.205346    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:58.648878 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:58.648847    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:05:58.649827 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:05:58.649808    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:06:07.443358 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:07.443323    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"
Sep 14 03:06:17.505784 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:17.505739    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs"]
Sep 14 03:06:17.508426 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:17.508406    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs"
Sep 14 03:06:17.511313 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:17.511294    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:06:17.517986 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:17.517941    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs"]
Sep 14 03:06:17.622937 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:17.622907    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bdbe281-c29b-41a0-b786-7dce32870dce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs\" (UID: \"6bdbe281-c29b-41a0-b786-7dce32870dce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs"
Sep 14 03:06:17.723859 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:17.723824    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bdbe281-c29b-41a0-b786-7dce32870dce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs\" (UID: \"6bdbe281-c29b-41a0-b786-7dce32870dce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs"
Sep 14 03:06:17.724847 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:17.724820    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bdbe281-c29b-41a0-b786-7dce32870dce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs\" (UID: \"6bdbe281-c29b-41a0-b786-7dce32870dce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs"
Sep 14 03:06:17.821965 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:17.821906    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:06:17.830276 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:17.830254    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs"
Sep 14 03:06:18.103706 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:18.103683    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs"]
Sep 14 03:06:18.105229 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:06:18.105166    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bdbe281_c29b_41a0_b786_7dce32870dce.slice/crio-be5888c8662538d90310f9697dd96b9a12642203e6f3214bf5322df1ea484bda WatchSource:0}: Error finding container be5888c8662538d90310f9697dd96b9a12642203e6f3214bf5322df1ea484bda: Status 404 returned error can't find the container with id be5888c8662538d90310f9697dd96b9a12642203e6f3214bf5322df1ea484bda
Sep 14 03:06:18.547659 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:18.547624    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs" event={"ID":"6bdbe281-c29b-41a0-b786-7dce32870dce","Type":"ContainerStarted","Data":"be5888c8662538d90310f9697dd96b9a12642203e6f3214bf5322df1ea484bda"}
Sep 14 03:06:20.559521 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:20.559478    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs" event={"ID":"6bdbe281-c29b-41a0-b786-7dce32870dce","Type":"ContainerStarted","Data":"ceebfb8b10525ee0c84afc893e039afac02fca69929e46f88ce38692a3bf3456"}
Sep 14 03:06:20.579635 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:20.579581    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs" podStartSLOduration=1.553298611 podStartE2EDuration="3.579566277s" totalImagesPullingTime="2.026267666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:06:20.578128792 +0000 UTC m=+843.364190706" watchObservedRunningTime="2026-09-14 03:06:20.579566277 +0000 UTC m=+843.365628190"
Sep 14 03:06:21.565772 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:21.565739    2579 generic.go:356] "Generic (PLEG): container finished" podID="6bdbe281-c29b-41a0-b786-7dce32870dce" containerID="ceebfb8b10525ee0c84afc893e039afac02fca69929e46f88ce38692a3bf3456" exitCode=0
Sep 14 03:06:21.566208 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:21.565826    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs" event={"ID":"6bdbe281-c29b-41a0-b786-7dce32870dce","Type":"ContainerDied","Data":"ceebfb8b10525ee0c84afc893e039afac02fca69929e46f88ce38692a3bf3456"}
Sep 14 03:06:22.711615 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:22.711593    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs"
Sep 14 03:06:22.874855 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:22.874773    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bdbe281-c29b-41a0-b786-7dce32870dce-maas-api-service-ca\") pod \"6bdbe281-c29b-41a0-b786-7dce32870dce\" (UID: \"6bdbe281-c29b-41a0-b786-7dce32870dce\") "
Sep 14 03:06:22.875115 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:22.875090    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bdbe281-c29b-41a0-b786-7dce32870dce-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6bdbe281-c29b-41a0-b786-7dce32870dce" (UID: "6bdbe281-c29b-41a0-b786-7dce32870dce"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:06:22.976259 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:22.976226    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bdbe281-c29b-41a0-b786-7dce32870dce-maas-api-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:06:23.576743 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:23.576713    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs"
Sep 14 03:06:23.576743 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:23.576737    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs" event={"ID":"6bdbe281-c29b-41a0-b786-7dce32870dce","Type":"ContainerDied","Data":"be5888c8662538d90310f9697dd96b9a12642203e6f3214bf5322df1ea484bda"}
Sep 14 03:06:23.576940 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:23.576782    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be5888c8662538d90310f9697dd96b9a12642203e6f3214bf5322df1ea484bda"
Sep 14 03:06:34.498192 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:34.498148    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"]
Sep 14 03:06:34.498935 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:34.498436    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q" podUID="277f486c-44dd-4e5c-b285-64e8ca2f15a6" containerName="payload-processing" containerID="cri-o://4e4c6d45773e1cf84472e2bec57edc9857320d78d833d150e0d3cf10b3ff8443" gracePeriod=30
Sep 14 03:06:34.640646 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:34.640613    2579 generic.go:356] "Generic (PLEG): container finished" podID="277f486c-44dd-4e5c-b285-64e8ca2f15a6" containerID="4e4c6d45773e1cf84472e2bec57edc9857320d78d833d150e0d3cf10b3ff8443" exitCode=0
Sep 14 03:06:34.640796 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:34.640683    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q" event={"ID":"277f486c-44dd-4e5c-b285-64e8ca2f15a6","Type":"ContainerDied","Data":"4e4c6d45773e1cf84472e2bec57edc9857320d78d833d150e0d3cf10b3ff8443"}
Sep 14 03:06:34.715008 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:34.714978    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs"]
Sep 14 03:06:34.721386 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:34.721362    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d8a641aa-xdkxs"]
Sep 14 03:06:34.754055 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:34.754008    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"
Sep 14 03:06:34.886644 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:34.886610    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/277f486c-44dd-4e5c-b285-64e8ca2f15a6-plugins-config-volume\") pod \"277f486c-44dd-4e5c-b285-64e8ca2f15a6\" (UID: \"277f486c-44dd-4e5c-b285-64e8ca2f15a6\") "
Sep 14 03:06:34.886825 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:34.886726    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zwl4\" (UniqueName: \"kubernetes.io/projected/277f486c-44dd-4e5c-b285-64e8ca2f15a6-kube-api-access-2zwl4\") pod \"277f486c-44dd-4e5c-b285-64e8ca2f15a6\" (UID: \"277f486c-44dd-4e5c-b285-64e8ca2f15a6\") "
Sep 14 03:06:34.887027 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:34.887000    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/277f486c-44dd-4e5c-b285-64e8ca2f15a6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "277f486c-44dd-4e5c-b285-64e8ca2f15a6" (UID: "277f486c-44dd-4e5c-b285-64e8ca2f15a6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:06:34.888887 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:34.888859    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/277f486c-44dd-4e5c-b285-64e8ca2f15a6-kube-api-access-2zwl4" (OuterVolumeSpecName: "kube-api-access-2zwl4") pod "277f486c-44dd-4e5c-b285-64e8ca2f15a6" (UID: "277f486c-44dd-4e5c-b285-64e8ca2f15a6"). InnerVolumeSpecName "kube-api-access-2zwl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:06:34.987954 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:34.987918    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zwl4\" (UniqueName: \"kubernetes.io/projected/277f486c-44dd-4e5c-b285-64e8ca2f15a6-kube-api-access-2zwl4\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:06:34.987954 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:34.987945    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/277f486c-44dd-4e5c-b285-64e8ca2f15a6-plugins-config-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:06:35.647247 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:35.647212    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q" event={"ID":"277f486c-44dd-4e5c-b285-64e8ca2f15a6","Type":"ContainerDied","Data":"1db9c06bbbb9eb46befbf2f4330e1cc751c98102758e099bd06ce119e1b9610c"}
Sep 14 03:06:35.647680 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:35.647300    2579 scope.go:117] "RemoveContainer" containerID="4e4c6d45773e1cf84472e2bec57edc9857320d78d833d150e0d3cf10b3ff8443"
Sep 14 03:06:35.647680 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:35.647381    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"
Sep 14 03:06:35.674436 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:35.674397    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"]
Sep 14 03:06:35.677347 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:35.677325    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d8a641aa-8b7fd7458-n9d7q"]
Sep 14 03:06:35.747250 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:35.747202    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="277f486c-44dd-4e5c-b285-64e8ca2f15a6" path="/var/lib/kubelet/pods/277f486c-44dd-4e5c-b285-64e8ca2f15a6/volumes"
Sep 14 03:06:35.747706 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:35.747685    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bdbe281-c29b-41a0-b786-7dce32870dce" path="/var/lib/kubelet/pods/6bdbe281-c29b-41a0-b786-7dce32870dce/volumes"
Sep 14 03:06:41.143047 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.143010    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"]
Sep 14 03:06:41.143449 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.143434    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="277f486c-44dd-4e5c-b285-64e8ca2f15a6" containerName="payload-processing"
Sep 14 03:06:41.143502 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.143451    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="277f486c-44dd-4e5c-b285-64e8ca2f15a6" containerName="payload-processing"
Sep 14 03:06:41.143502 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.143468    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bdbe281-c29b-41a0-b786-7dce32870dce" containerName="revoke-keys"
Sep 14 03:06:41.143502 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.143473    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bdbe281-c29b-41a0-b786-7dce32870dce" containerName="revoke-keys"
Sep 14 03:06:41.143600 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.143535    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="277f486c-44dd-4e5c-b285-64e8ca2f15a6" containerName="payload-processing"
Sep 14 03:06:41.143600 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.143544    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bdbe281-c29b-41a0-b786-7dce32870dce" containerName="revoke-keys"
Sep 14 03:06:41.146617 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.146600    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"
Sep 14 03:06:41.150354 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.150332    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-65e77699\""
Sep 14 03:06:41.150527 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.150506    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-65e77699-dockercfg-f2nwt\""
Sep 14 03:06:41.159050 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.159030    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"]
Sep 14 03:06:41.242736 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.242651    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/698f38e3-379c-4d0e-87e5-0d873848d173-plugins-config-volume\") pod \"payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d\" (UID: \"698f38e3-379c-4d0e-87e5-0d873848d173\") " pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"
Sep 14 03:06:41.242933 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.242894    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4xvf\" (UniqueName: \"kubernetes.io/projected/698f38e3-379c-4d0e-87e5-0d873848d173-kube-api-access-h4xvf\") pod \"payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d\" (UID: \"698f38e3-379c-4d0e-87e5-0d873848d173\") " pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"
Sep 14 03:06:41.343712 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.343674    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/698f38e3-379c-4d0e-87e5-0d873848d173-plugins-config-volume\") pod \"payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d\" (UID: \"698f38e3-379c-4d0e-87e5-0d873848d173\") " pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"
Sep 14 03:06:41.343908 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.343719    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4xvf\" (UniqueName: \"kubernetes.io/projected/698f38e3-379c-4d0e-87e5-0d873848d173-kube-api-access-h4xvf\") pod \"payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d\" (UID: \"698f38e3-379c-4d0e-87e5-0d873848d173\") " pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"
Sep 14 03:06:41.344465 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.344439    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/698f38e3-379c-4d0e-87e5-0d873848d173-plugins-config-volume\") pod \"payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d\" (UID: \"698f38e3-379c-4d0e-87e5-0d873848d173\") " pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"
Sep 14 03:06:41.353145 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.353122    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4xvf\" (UniqueName: \"kubernetes.io/projected/698f38e3-379c-4d0e-87e5-0d873848d173-kube-api-access-h4xvf\") pod \"payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d\" (UID: \"698f38e3-379c-4d0e-87e5-0d873848d173\") " pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"
Sep 14 03:06:41.457115 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.457026    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"
Sep 14 03:06:41.746573 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:41.746547    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"]
Sep 14 03:06:41.747848 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:06:41.747819    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod698f38e3_379c_4d0e_87e5_0d873848d173.slice/crio-e5a4a32ed2143cd84838a262f7f4d337b5e50ec28072fb7c582bfaa1dad4632e WatchSource:0}: Error finding container e5a4a32ed2143cd84838a262f7f4d337b5e50ec28072fb7c582bfaa1dad4632e: Status 404 returned error can't find the container with id e5a4a32ed2143cd84838a262f7f4d337b5e50ec28072fb7c582bfaa1dad4632e
Sep 14 03:06:42.684454 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:42.684412    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d" event={"ID":"698f38e3-379c-4d0e-87e5-0d873848d173","Type":"ContainerStarted","Data":"b2e20813570957de1c392ef05d3cf2101e5aab04999babba9ed30d6f361ec145"}
Sep 14 03:06:42.684845 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:42.684476    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d" event={"ID":"698f38e3-379c-4d0e-87e5-0d873848d173","Type":"ContainerStarted","Data":"e5a4a32ed2143cd84838a262f7f4d337b5e50ec28072fb7c582bfaa1dad4632e"}
Sep 14 03:06:42.684845 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:42.684503    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"
Sep 14 03:06:42.733347 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:42.733295    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d" podStartSLOduration=1.733278929 podStartE2EDuration="1.733278929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:06:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:06:42.73145772 +0000 UTC m=+865.517519634" watchObservedRunningTime="2026-09-14 03:06:42.733278929 +0000 UTC m=+865.519340843"
Sep 14 03:06:53.691874 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:06:53.691843    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"
Sep 14 03:07:04.405918 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:04.405871    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-65e77699-dqnk6"]
Sep 14 03:07:04.412960 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:04.412938    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-65e77699-dqnk6"
Sep 14 03:07:04.416638 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:04.416606    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:07:04.420134 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:04.420110    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-65e77699-dqnk6"]
Sep 14 03:07:04.552833 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:04.552797    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06c99388-ce0f-48e0-8414-863f9c52a6b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-65e77699-dqnk6\" (UID: \"06c99388-ce0f-48e0-8414-863f9c52a6b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-65e77699-dqnk6"
Sep 14 03:07:04.653536 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:04.653507    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06c99388-ce0f-48e0-8414-863f9c52a6b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-65e77699-dqnk6\" (UID: \"06c99388-ce0f-48e0-8414-863f9c52a6b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-65e77699-dqnk6"
Sep 14 03:07:04.654095 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:04.654077    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06c99388-ce0f-48e0-8414-863f9c52a6b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-65e77699-dqnk6\" (UID: \"06c99388-ce0f-48e0-8414-863f9c52a6b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-65e77699-dqnk6"
Sep 14 03:07:04.743956 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:04.743929    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-65e77699-dqnk6"
Sep 14 03:07:05.027505 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:05.027476    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-65e77699-dqnk6"]
Sep 14 03:07:05.028771 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:07:05.028740    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06c99388_ce0f_48e0_8414_863f9c52a6b7.slice/crio-497e356bc0afb85251971bbc142bc323bd9eee6a1be0e2646dcc5b2659a52286 WatchSource:0}: Error finding container 497e356bc0afb85251971bbc142bc323bd9eee6a1be0e2646dcc5b2659a52286: Status 404 returned error can't find the container with id 497e356bc0afb85251971bbc142bc323bd9eee6a1be0e2646dcc5b2659a52286
Sep 14 03:07:05.816892 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:05.816856    2579 generic.go:356] "Generic (PLEG): container finished" podID="06c99388-ce0f-48e0-8414-863f9c52a6b7" containerID="88ac0d9ce27b1e677ab4b378d507c27842009a7b19e9b70ce8cbb756317a053d" exitCode=0
Sep 14 03:07:05.817271 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:05.816945    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-65e77699-dqnk6" event={"ID":"06c99388-ce0f-48e0-8414-863f9c52a6b7","Type":"ContainerDied","Data":"88ac0d9ce27b1e677ab4b378d507c27842009a7b19e9b70ce8cbb756317a053d"}
Sep 14 03:07:05.817271 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:05.816992    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-65e77699-dqnk6" event={"ID":"06c99388-ce0f-48e0-8414-863f9c52a6b7","Type":"ContainerStarted","Data":"497e356bc0afb85251971bbc142bc323bd9eee6a1be0e2646dcc5b2659a52286"}
Sep 14 03:07:06.957047 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:06.957022    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-65e77699-dqnk6"
Sep 14 03:07:07.075924 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:07.075844    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06c99388-ce0f-48e0-8414-863f9c52a6b7-maas-api-service-ca\") pod \"06c99388-ce0f-48e0-8414-863f9c52a6b7\" (UID: \"06c99388-ce0f-48e0-8414-863f9c52a6b7\") "
Sep 14 03:07:07.076192 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:07.076156    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06c99388-ce0f-48e0-8414-863f9c52a6b7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "06c99388-ce0f-48e0-8414-863f9c52a6b7" (UID: "06c99388-ce0f-48e0-8414-863f9c52a6b7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:07:07.176561 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:07.176531    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/06c99388-ce0f-48e0-8414-863f9c52a6b7-maas-api-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:07.827461 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:07.827431    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-65e77699-dqnk6" event={"ID":"06c99388-ce0f-48e0-8414-863f9c52a6b7","Type":"ContainerDied","Data":"497e356bc0afb85251971bbc142bc323bd9eee6a1be0e2646dcc5b2659a52286"}
Sep 14 03:07:07.827634 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:07.827466    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-65e77699-dqnk6"
Sep 14 03:07:07.827634 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:07.827477    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="497e356bc0afb85251971bbc142bc323bd9eee6a1be0e2646dcc5b2659a52286"
Sep 14 03:07:10.855002 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:10.854916    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"]
Sep 14 03:07:10.855502 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:10.855193    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d" podUID="698f38e3-379c-4d0e-87e5-0d873848d173" containerName="payload-processing" containerID="cri-o://b2e20813570957de1c392ef05d3cf2101e5aab04999babba9ed30d6f361ec145" gracePeriod=30
Sep 14 03:07:11.022926 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.022893    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-65e77699-dqnk6"]
Sep 14 03:07:11.026291 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.026149    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-65e77699-dqnk6"]
Sep 14 03:07:11.106712 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.106657    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"
Sep 14 03:07:11.211865 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.211827    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/698f38e3-379c-4d0e-87e5-0d873848d173-plugins-config-volume\") pod \"698f38e3-379c-4d0e-87e5-0d873848d173\" (UID: \"698f38e3-379c-4d0e-87e5-0d873848d173\") "
Sep 14 03:07:11.212085 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.211876    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4xvf\" (UniqueName: \"kubernetes.io/projected/698f38e3-379c-4d0e-87e5-0d873848d173-kube-api-access-h4xvf\") pod \"698f38e3-379c-4d0e-87e5-0d873848d173\" (UID: \"698f38e3-379c-4d0e-87e5-0d873848d173\") "
Sep 14 03:07:11.212602 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.212573    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/698f38e3-379c-4d0e-87e5-0d873848d173-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "698f38e3-379c-4d0e-87e5-0d873848d173" (UID: "698f38e3-379c-4d0e-87e5-0d873848d173"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:07:11.215146 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.215114    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/698f38e3-379c-4d0e-87e5-0d873848d173-kube-api-access-h4xvf" (OuterVolumeSpecName: "kube-api-access-h4xvf") pod "698f38e3-379c-4d0e-87e5-0d873848d173" (UID: "698f38e3-379c-4d0e-87e5-0d873848d173"). InnerVolumeSpecName "kube-api-access-h4xvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:07:11.313333 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.313306    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/698f38e3-379c-4d0e-87e5-0d873848d173-plugins-config-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:11.313333 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.313326    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4xvf\" (UniqueName: \"kubernetes.io/projected/698f38e3-379c-4d0e-87e5-0d873848d173-kube-api-access-h4xvf\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:11.743729 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.743695    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06c99388-ce0f-48e0-8414-863f9c52a6b7" path="/var/lib/kubelet/pods/06c99388-ce0f-48e0-8414-863f9c52a6b7/volumes"
Sep 14 03:07:11.849234 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.849199    2579 generic.go:356] "Generic (PLEG): container finished" podID="698f38e3-379c-4d0e-87e5-0d873848d173" containerID="b2e20813570957de1c392ef05d3cf2101e5aab04999babba9ed30d6f361ec145" exitCode=0
Sep 14 03:07:11.849385 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.849251    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"
Sep 14 03:07:11.849385 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.849261    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d" event={"ID":"698f38e3-379c-4d0e-87e5-0d873848d173","Type":"ContainerDied","Data":"b2e20813570957de1c392ef05d3cf2101e5aab04999babba9ed30d6f361ec145"}
Sep 14 03:07:11.849385 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.849311    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d" event={"ID":"698f38e3-379c-4d0e-87e5-0d873848d173","Type":"ContainerDied","Data":"e5a4a32ed2143cd84838a262f7f4d337b5e50ec28072fb7c582bfaa1dad4632e"}
Sep 14 03:07:11.849385 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.849343    2579 scope.go:117] "RemoveContainer" containerID="b2e20813570957de1c392ef05d3cf2101e5aab04999babba9ed30d6f361ec145"
Sep 14 03:07:11.858373 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.858221    2579 scope.go:117] "RemoveContainer" containerID="b2e20813570957de1c392ef05d3cf2101e5aab04999babba9ed30d6f361ec145"
Sep 14 03:07:11.858587 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:07:11.858471    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2e20813570957de1c392ef05d3cf2101e5aab04999babba9ed30d6f361ec145\": container with ID starting with b2e20813570957de1c392ef05d3cf2101e5aab04999babba9ed30d6f361ec145 not found: ID does not exist" containerID="b2e20813570957de1c392ef05d3cf2101e5aab04999babba9ed30d6f361ec145"
Sep 14 03:07:11.858587 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.858496    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2e20813570957de1c392ef05d3cf2101e5aab04999babba9ed30d6f361ec145"} err="failed to get container status \"b2e20813570957de1c392ef05d3cf2101e5aab04999babba9ed30d6f361ec145\": rpc error: code = NotFound desc = could not find container \"b2e20813570957de1c392ef05d3cf2101e5aab04999babba9ed30d6f361ec145\": container with ID starting with b2e20813570957de1c392ef05d3cf2101e5aab04999babba9ed30d6f361ec145 not found: ID does not exist"
Sep 14 03:07:11.870349 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.870319    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"]
Sep 14 03:07:11.878890 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:11.878869    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-65e77699-5cf59468c5-xcx4d"]
Sep 14 03:07:12.886651 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:12.886617    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"]
Sep 14 03:07:12.887056 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:12.887043    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="698f38e3-379c-4d0e-87e5-0d873848d173" containerName="payload-processing"
Sep 14 03:07:12.887099 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:12.887058    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="698f38e3-379c-4d0e-87e5-0d873848d173" containerName="payload-processing"
Sep 14 03:07:12.887099 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:12.887072    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06c99388-ce0f-48e0-8414-863f9c52a6b7" containerName="revoke-keys"
Sep 14 03:07:12.887099 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:12.887077    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="06c99388-ce0f-48e0-8414-863f9c52a6b7" containerName="revoke-keys"
Sep 14 03:07:12.887226 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:12.887153    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="06c99388-ce0f-48e0-8414-863f9c52a6b7" containerName="revoke-keys"
Sep 14 03:07:12.887226 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:12.887161    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="698f38e3-379c-4d0e-87e5-0d873848d173" containerName="payload-processing"
Sep 14 03:07:12.890004 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:12.889987    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:12.893391 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:12.893374    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-fb76f4c2-openshift-default-dockercfg-66vvt\""
Sep 14 03:07:12.904715 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:12.904694    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"]
Sep 14 03:07:13.028298 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.028266    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-workload-certs\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.028487 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.028307    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-envoy\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.028487 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.028331    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-data\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.028487 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.028412    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zplf\" (UniqueName: \"kubernetes.io/projected/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-kube-api-access-2zplf\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.028648 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.028510    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-credential-socket\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.028648 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.028554    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-workload-socket\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.028648 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.028592    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-podinfo\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.028758 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.028669    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-token\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.028758 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.028700    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istiod-ca-cert\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.129142 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.129106    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-podinfo\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.129358 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.129152    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-token\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.129358 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.129201    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istiod-ca-cert\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.129358 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.129220    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-workload-certs\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.129358 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.129253    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-envoy\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.129358 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.129286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-data\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.129358 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.129321    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zplf\" (UniqueName: \"kubernetes.io/projected/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-kube-api-access-2zplf\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.129670 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.129400    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-credential-socket\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.129670 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.129425    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-workload-socket\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.129776 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.129758    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-workload-socket\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.129884 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.129865    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-workload-certs\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.130019 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.129865    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-data\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.130087 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.130020    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-credential-socket\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.130153 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.130139    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istiod-ca-cert\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.131834 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.131812    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-envoy\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.131955 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.131933    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-podinfo\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.138141 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.138088    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-token\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.138141 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.138119    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zplf\" (UniqueName: \"kubernetes.io/projected/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-kube-api-access-2zplf\") pod \"e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") " pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.201525 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.201300    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:13.483828 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.483805    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"]
Sep 14 03:07:13.485669 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:07:13.485641    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2971ed6e_54f9_46e1_b5b3_b740fcbf3170.slice/crio-7e2a6392974844b346d3e92c0ccee3fc0d58bd051524148a1c1ab90261d5b790 WatchSource:0}: Error finding container 7e2a6392974844b346d3e92c0ccee3fc0d58bd051524148a1c1ab90261d5b790: Status 404 returned error can't find the container with id 7e2a6392974844b346d3e92c0ccee3fc0d58bd051524148a1c1ab90261d5b790
Sep 14 03:07:13.488099 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.488060    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:07:13.488219 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.488149    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:07:13.488267 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.488249    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:07:13.744907 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.744870    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="698f38e3-379c-4d0e-87e5-0d873848d173" path="/var/lib/kubelet/pods/698f38e3-379c-4d0e-87e5-0d873848d173/volumes"
Sep 14 03:07:13.862389 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.862350    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr" event={"ID":"2971ed6e-54f9-46e1-b5b3-b740fcbf3170","Type":"ContainerStarted","Data":"93222743da3f93a2785d451011abe1039a4bd48bf6c1773fd0632f60433090c7"}
Sep 14 03:07:13.862563 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.862418    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr" event={"ID":"2971ed6e-54f9-46e1-b5b3-b740fcbf3170","Type":"ContainerStarted","Data":"7e2a6392974844b346d3e92c0ccee3fc0d58bd051524148a1c1ab90261d5b790"}
Sep 14 03:07:13.886146 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:13.886091    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr" podStartSLOduration=1.886073148 podStartE2EDuration="1.886073148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:07:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:07:13.88309946 +0000 UTC m=+896.669161376" watchObservedRunningTime="2026-09-14 03:07:13.886073148 +0000 UTC m=+896.672135063"
Sep 14 03:07:14.202206 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:14.202100    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:14.207515 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:14.207483    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:14.867911 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:14.867886    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:14.868766 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:14.868747    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:19.102102 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.102072    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"]
Sep 14 03:07:19.107635 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.107617    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"
Sep 14 03:07:19.111142 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.111077    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-fb76f4c2\""
Sep 14 03:07:19.111297 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.111240    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-fb76f4c2-dockercfg-hmncl\""
Sep 14 03:07:19.117590 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.117569    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"]
Sep 14 03:07:19.290637 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.290602    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69683372-d185-4d06-b375-da9936e7d07b-plugins-config-volume\") pod \"payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk\" (UID: \"69683372-d185-4d06-b375-da9936e7d07b\") " pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"
Sep 14 03:07:19.290835 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.290652    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnvgk\" (UniqueName: \"kubernetes.io/projected/69683372-d185-4d06-b375-da9936e7d07b-kube-api-access-lnvgk\") pod \"payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk\" (UID: \"69683372-d185-4d06-b375-da9936e7d07b\") " pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"
Sep 14 03:07:19.391482 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.391400    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69683372-d185-4d06-b375-da9936e7d07b-plugins-config-volume\") pod \"payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk\" (UID: \"69683372-d185-4d06-b375-da9936e7d07b\") " pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"
Sep 14 03:07:19.391482 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.391450    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnvgk\" (UniqueName: \"kubernetes.io/projected/69683372-d185-4d06-b375-da9936e7d07b-kube-api-access-lnvgk\") pod \"payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk\" (UID: \"69683372-d185-4d06-b375-da9936e7d07b\") " pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"
Sep 14 03:07:19.392089 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.392067    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69683372-d185-4d06-b375-da9936e7d07b-plugins-config-volume\") pod \"payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk\" (UID: \"69683372-d185-4d06-b375-da9936e7d07b\") " pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"
Sep 14 03:07:19.402861 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.402831    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnvgk\" (UniqueName: \"kubernetes.io/projected/69683372-d185-4d06-b375-da9936e7d07b-kube-api-access-lnvgk\") pod \"payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk\" (UID: \"69683372-d185-4d06-b375-da9936e7d07b\") " pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"
Sep 14 03:07:19.419812 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.419785    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"
Sep 14 03:07:19.694067 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.694044    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"]
Sep 14 03:07:19.695998 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:07:19.695968    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69683372_d185_4d06_b375_da9936e7d07b.slice/crio-2b32d9f077cd575da61dc77848e539306292ec7351bf9f8bbf07e7c0419cc61f WatchSource:0}: Error finding container 2b32d9f077cd575da61dc77848e539306292ec7351bf9f8bbf07e7c0419cc61f: Status 404 returned error can't find the container with id 2b32d9f077cd575da61dc77848e539306292ec7351bf9f8bbf07e7c0419cc61f
Sep 14 03:07:19.893734 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.893701    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk" event={"ID":"69683372-d185-4d06-b375-da9936e7d07b","Type":"ContainerStarted","Data":"1417d3a7f96e67bb5acf0dbac53d24939d9282bcd4eb2501901c572cbaca9308"}
Sep 14 03:07:19.893934 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.893776    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"
Sep 14 03:07:19.893934 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.893801    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk" event={"ID":"69683372-d185-4d06-b375-da9936e7d07b","Type":"ContainerStarted","Data":"2b32d9f077cd575da61dc77848e539306292ec7351bf9f8bbf07e7c0419cc61f"}
Sep 14 03:07:19.913193 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:19.913127    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk" podStartSLOduration=0.913110883 podStartE2EDuration="913.110883ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:07:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:07:19.911948186 +0000 UTC m=+902.698010100" watchObservedRunningTime="2026-09-14 03:07:19.913110883 +0000 UTC m=+902.699172797"
Sep 14 03:07:23.052288 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.052253    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"]
Sep 14 03:07:23.052753 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.052530    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w" podUID="58162ed7-956f-4e12-9059-6d6139b1cc12" containerName="maas-api" containerID="cri-o://40cd3af0fab2f541ef9ba0e84030f8ee4bd68b83c47571ab30f43f55ff5c2e39" gracePeriod=30
Sep 14 03:07:23.114917 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.114888    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"]
Sep 14 03:07:23.115219 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.115189    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f" podUID="0bbd8c1f-18eb-4dd3-be45-ebe4195a4668" containerName="payload-processing" containerID="cri-o://ec4d03d4e973d5c97011c21d9085d33c0ba3b7f9d037f42d57fe1dcdd19fb177" gracePeriod=30
Sep 14 03:07:23.409167 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.409092    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"
Sep 14 03:07:23.411956 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.411928    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:07:23.432999 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.432980    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58162ed7-956f-4e12-9059-6d6139b1cc12-openshift-service-ca\") pod \"58162ed7-956f-4e12-9059-6d6139b1cc12\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") "
Sep 14 03:07:23.433096 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.433008    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bbd8c1f-18eb-4dd3-be45-ebe4195a4668-plugins-config-volume\") pod \"0bbd8c1f-18eb-4dd3-be45-ebe4195a4668\" (UID: \"0bbd8c1f-18eb-4dd3-be45-ebe4195a4668\") "
Sep 14 03:07:23.433096 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.433045    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcmc7\" (UniqueName: \"kubernetes.io/projected/58162ed7-956f-4e12-9059-6d6139b1cc12-kube-api-access-bcmc7\") pod \"58162ed7-956f-4e12-9059-6d6139b1cc12\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") "
Sep 14 03:07:23.433096 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.433066    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2cwd\" (UniqueName: \"kubernetes.io/projected/0bbd8c1f-18eb-4dd3-be45-ebe4195a4668-kube-api-access-c2cwd\") pod \"0bbd8c1f-18eb-4dd3-be45-ebe4195a4668\" (UID: \"0bbd8c1f-18eb-4dd3-be45-ebe4195a4668\") "
Sep 14 03:07:23.433096 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.433093    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58162ed7-956f-4e12-9059-6d6139b1cc12-maas-api-tls\") pod \"58162ed7-956f-4e12-9059-6d6139b1cc12\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") "
Sep 14 03:07:23.433282 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.433138    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58162ed7-956f-4e12-9059-6d6139b1cc12-metrics-certs\") pod \"58162ed7-956f-4e12-9059-6d6139b1cc12\" (UID: \"58162ed7-956f-4e12-9059-6d6139b1cc12\") "
Sep 14 03:07:23.433446 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.433410    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58162ed7-956f-4e12-9059-6d6139b1cc12-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "58162ed7-956f-4e12-9059-6d6139b1cc12" (UID: "58162ed7-956f-4e12-9059-6d6139b1cc12"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:07:23.433446 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.433409    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bbd8c1f-18eb-4dd3-be45-ebe4195a4668-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0bbd8c1f-18eb-4dd3-be45-ebe4195a4668" (UID: "0bbd8c1f-18eb-4dd3-be45-ebe4195a4668"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:07:23.433598 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.433511    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58162ed7-956f-4e12-9059-6d6139b1cc12-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:23.433598 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.433531    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bbd8c1f-18eb-4dd3-be45-ebe4195a4668-plugins-config-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:23.435405 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.435380    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bbd8c1f-18eb-4dd3-be45-ebe4195a4668-kube-api-access-c2cwd" (OuterVolumeSpecName: "kube-api-access-c2cwd") pod "0bbd8c1f-18eb-4dd3-be45-ebe4195a4668" (UID: "0bbd8c1f-18eb-4dd3-be45-ebe4195a4668"). InnerVolumeSpecName "kube-api-access-c2cwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:07:23.435496 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.435431    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58162ed7-956f-4e12-9059-6d6139b1cc12-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "58162ed7-956f-4e12-9059-6d6139b1cc12" (UID: "58162ed7-956f-4e12-9059-6d6139b1cc12"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:07:23.435541 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.435495    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58162ed7-956f-4e12-9059-6d6139b1cc12-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "58162ed7-956f-4e12-9059-6d6139b1cc12" (UID: "58162ed7-956f-4e12-9059-6d6139b1cc12"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:07:23.435541 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.435526    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58162ed7-956f-4e12-9059-6d6139b1cc12-kube-api-access-bcmc7" (OuterVolumeSpecName: "kube-api-access-bcmc7") pod "58162ed7-956f-4e12-9059-6d6139b1cc12" (UID: "58162ed7-956f-4e12-9059-6d6139b1cc12"). InnerVolumeSpecName "kube-api-access-bcmc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:07:23.534116 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.534085    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcmc7\" (UniqueName: \"kubernetes.io/projected/58162ed7-956f-4e12-9059-6d6139b1cc12-kube-api-access-bcmc7\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:23.534116 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.534111    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2cwd\" (UniqueName: \"kubernetes.io/projected/0bbd8c1f-18eb-4dd3-be45-ebe4195a4668-kube-api-access-c2cwd\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:23.534116 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.534121    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58162ed7-956f-4e12-9059-6d6139b1cc12-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:23.534348 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.534130    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58162ed7-956f-4e12-9059-6d6139b1cc12-metrics-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:23.915429 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.915394    2579 generic.go:356] "Generic (PLEG): container finished" podID="58162ed7-956f-4e12-9059-6d6139b1cc12" containerID="40cd3af0fab2f541ef9ba0e84030f8ee4bd68b83c47571ab30f43f55ff5c2e39" exitCode=0
Sep 14 03:07:23.915607 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.915449    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"
Sep 14 03:07:23.915607 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.915484    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w" event={"ID":"58162ed7-956f-4e12-9059-6d6139b1cc12","Type":"ContainerDied","Data":"40cd3af0fab2f541ef9ba0e84030f8ee4bd68b83c47571ab30f43f55ff5c2e39"}
Sep 14 03:07:23.915607 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.915555    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w" event={"ID":"58162ed7-956f-4e12-9059-6d6139b1cc12","Type":"ContainerDied","Data":"3a6b7aa05c1f4fd34395a1665e18d95cddf7b2dddda323e15d777f3dcb1e8bf2"}
Sep 14 03:07:23.915607 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.915578    2579 scope.go:117] "RemoveContainer" containerID="40cd3af0fab2f541ef9ba0e84030f8ee4bd68b83c47571ab30f43f55ff5c2e39"
Sep 14 03:07:23.917134 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.917108    2579 generic.go:356] "Generic (PLEG): container finished" podID="0bbd8c1f-18eb-4dd3-be45-ebe4195a4668" containerID="ec4d03d4e973d5c97011c21d9085d33c0ba3b7f9d037f42d57fe1dcdd19fb177" exitCode=0
Sep 14 03:07:23.917281 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.917168    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"
Sep 14 03:07:23.917281 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.917199    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f" event={"ID":"0bbd8c1f-18eb-4dd3-be45-ebe4195a4668","Type":"ContainerDied","Data":"ec4d03d4e973d5c97011c21d9085d33c0ba3b7f9d037f42d57fe1dcdd19fb177"}
Sep 14 03:07:23.917281 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.917241    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f" event={"ID":"0bbd8c1f-18eb-4dd3-be45-ebe4195a4668","Type":"ContainerDied","Data":"1090078eb6f5bc7dc64edef94402fa7b918cb3ed242011482ec99224f0f7efd4"}
Sep 14 03:07:23.925010 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.924995    2579 scope.go:117] "RemoveContainer" containerID="40cd3af0fab2f541ef9ba0e84030f8ee4bd68b83c47571ab30f43f55ff5c2e39"
Sep 14 03:07:23.925269 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:07:23.925250    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40cd3af0fab2f541ef9ba0e84030f8ee4bd68b83c47571ab30f43f55ff5c2e39\": container with ID starting with 40cd3af0fab2f541ef9ba0e84030f8ee4bd68b83c47571ab30f43f55ff5c2e39 not found: ID does not exist" containerID="40cd3af0fab2f541ef9ba0e84030f8ee4bd68b83c47571ab30f43f55ff5c2e39"
Sep 14 03:07:23.925342 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.925275    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40cd3af0fab2f541ef9ba0e84030f8ee4bd68b83c47571ab30f43f55ff5c2e39"} err="failed to get container status \"40cd3af0fab2f541ef9ba0e84030f8ee4bd68b83c47571ab30f43f55ff5c2e39\": rpc error: code = NotFound desc = could not find container \"40cd3af0fab2f541ef9ba0e84030f8ee4bd68b83c47571ab30f43f55ff5c2e39\": container with ID starting with 40cd3af0fab2f541ef9ba0e84030f8ee4bd68b83c47571ab30f43f55ff5c2e39 not found: ID does not exist"
Sep 14 03:07:23.925342 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.925289    2579 scope.go:117] "RemoveContainer" containerID="ec4d03d4e973d5c97011c21d9085d33c0ba3b7f9d037f42d57fe1dcdd19fb177"
Sep 14 03:07:23.933513 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.933495    2579 scope.go:117] "RemoveContainer" containerID="ec4d03d4e973d5c97011c21d9085d33c0ba3b7f9d037f42d57fe1dcdd19fb177"
Sep 14 03:07:23.933858 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:07:23.933830    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec4d03d4e973d5c97011c21d9085d33c0ba3b7f9d037f42d57fe1dcdd19fb177\": container with ID starting with ec4d03d4e973d5c97011c21d9085d33c0ba3b7f9d037f42d57fe1dcdd19fb177 not found: ID does not exist" containerID="ec4d03d4e973d5c97011c21d9085d33c0ba3b7f9d037f42d57fe1dcdd19fb177"
Sep 14 03:07:23.933952 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.933868    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec4d03d4e973d5c97011c21d9085d33c0ba3b7f9d037f42d57fe1dcdd19fb177"} err="failed to get container status \"ec4d03d4e973d5c97011c21d9085d33c0ba3b7f9d037f42d57fe1dcdd19fb177\": rpc error: code = NotFound desc = could not find container \"ec4d03d4e973d5c97011c21d9085d33c0ba3b7f9d037f42d57fe1dcdd19fb177\": container with ID starting with ec4d03d4e973d5c97011c21d9085d33c0ba3b7f9d037f42d57fe1dcdd19fb177 not found: ID does not exist"
Sep 14 03:07:23.934301 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.934281    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"]
Sep 14 03:07:23.940517 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.940495    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-ab5750-688ddd6bc5-scq4w"]
Sep 14 03:07:23.955832 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.955813    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"]
Sep 14 03:07:23.960009 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:23.959989    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-ab5750-6f8df94989-kx65f"]
Sep 14 03:07:25.743808 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:25.743775    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bbd8c1f-18eb-4dd3-be45-ebe4195a4668" path="/var/lib/kubelet/pods/0bbd8c1f-18eb-4dd3-be45-ebe4195a4668/volumes"
Sep 14 03:07:25.744224 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:25.744209    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58162ed7-956f-4e12-9059-6d6139b1cc12" path="/var/lib/kubelet/pods/58162ed7-956f-4e12-9059-6d6139b1cc12/volumes"
Sep 14 03:07:29.875074 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:29.875044    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq"]
Sep 14 03:07:29.875492 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:29.875427    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bbd8c1f-18eb-4dd3-be45-ebe4195a4668" containerName="payload-processing"
Sep 14 03:07:29.875492 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:29.875438    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bbd8c1f-18eb-4dd3-be45-ebe4195a4668" containerName="payload-processing"
Sep 14 03:07:29.875492 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:29.875451    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58162ed7-956f-4e12-9059-6d6139b1cc12" containerName="maas-api"
Sep 14 03:07:29.875492 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:29.875456    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="58162ed7-956f-4e12-9059-6d6139b1cc12" containerName="maas-api"
Sep 14 03:07:29.875635 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:29.875523    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bbd8c1f-18eb-4dd3-be45-ebe4195a4668" containerName="payload-processing"
Sep 14 03:07:29.875635 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:29.875535    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="58162ed7-956f-4e12-9059-6d6139b1cc12" containerName="maas-api"
Sep 14 03:07:29.879936 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:29.879919    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq"
Sep 14 03:07:29.882672 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:29.882650    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:07:29.882795 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:29.882737    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:07:29.888499 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:29.888447    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq"]
Sep 14 03:07:29.988674 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:29.988642    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d902dcec-45c9-4ea8-ae12-ed6750ce2e9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq\" (UID: \"d902dcec-45c9-4ea8-ae12-ed6750ce2e9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq"
Sep 14 03:07:30.089099 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:30.089058    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d902dcec-45c9-4ea8-ae12-ed6750ce2e9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq\" (UID: \"d902dcec-45c9-4ea8-ae12-ed6750ce2e9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq"
Sep 14 03:07:30.089639 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:30.089620    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d902dcec-45c9-4ea8-ae12-ed6750ce2e9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq\" (UID: \"d902dcec-45c9-4ea8-ae12-ed6750ce2e9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq"
Sep 14 03:07:30.190702 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:30.190615    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq"
Sep 14 03:07:30.474750 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:30.474679    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq"]
Sep 14 03:07:30.478822 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:07:30.478792    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd902dcec_45c9_4ea8_ae12_ed6750ce2e9e.slice/crio-52e6ab110556857f19de7514b134407f7d2d507cfab7a938025acb0dae929881 WatchSource:0}: Error finding container 52e6ab110556857f19de7514b134407f7d2d507cfab7a938025acb0dae929881: Status 404 returned error can't find the container with id 52e6ab110556857f19de7514b134407f7d2d507cfab7a938025acb0dae929881
Sep 14 03:07:30.903280 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:30.902347    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"
Sep 14 03:07:30.956069 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:30.956033    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq" event={"ID":"d902dcec-45c9-4ea8-ae12-ed6750ce2e9e","Type":"ContainerStarted","Data":"afddc7a1185ea91d1cf23af2a1c105200f3bec824815a21591fcaf790b3ff546"}
Sep 14 03:07:30.956264 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:30.956100    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq" event={"ID":"d902dcec-45c9-4ea8-ae12-ed6750ce2e9e","Type":"ContainerStarted","Data":"52e6ab110556857f19de7514b134407f7d2d507cfab7a938025acb0dae929881"}
Sep 14 03:07:30.977081 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:30.977037    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq" podStartSLOduration=1.97702267 podStartE2EDuration="1.97702267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:07:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:07:30.976135882 +0000 UTC m=+913.762197797" watchObservedRunningTime="2026-09-14 03:07:30.97702267 +0000 UTC m=+913.763084583"
Sep 14 03:07:31.962588 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:31.962549    2579 generic.go:356] "Generic (PLEG): container finished" podID="d902dcec-45c9-4ea8-ae12-ed6750ce2e9e" containerID="afddc7a1185ea91d1cf23af2a1c105200f3bec824815a21591fcaf790b3ff546" exitCode=0
Sep 14 03:07:31.963022 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:31.962620    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq" event={"ID":"d902dcec-45c9-4ea8-ae12-ed6750ce2e9e","Type":"ContainerDied","Data":"afddc7a1185ea91d1cf23af2a1c105200f3bec824815a21591fcaf790b3ff546"}
Sep 14 03:07:33.147930 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:33.147900    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq"
Sep 14 03:07:33.229508 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:33.229430    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d902dcec-45c9-4ea8-ae12-ed6750ce2e9e-maas-api-service-ca\") pod \"d902dcec-45c9-4ea8-ae12-ed6750ce2e9e\" (UID: \"d902dcec-45c9-4ea8-ae12-ed6750ce2e9e\") "
Sep 14 03:07:33.229852 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:33.229831    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d902dcec-45c9-4ea8-ae12-ed6750ce2e9e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d902dcec-45c9-4ea8-ae12-ed6750ce2e9e" (UID: "d902dcec-45c9-4ea8-ae12-ed6750ce2e9e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:07:33.331075 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:33.331043    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d902dcec-45c9-4ea8-ae12-ed6750ce2e9e-maas-api-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:33.974311 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:33.974278    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq"
Sep 14 03:07:33.974651 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:33.974288    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq" event={"ID":"d902dcec-45c9-4ea8-ae12-ed6750ce2e9e","Type":"ContainerDied","Data":"52e6ab110556857f19de7514b134407f7d2d507cfab7a938025acb0dae929881"}
Sep 14 03:07:33.974651 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:33.974401    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52e6ab110556857f19de7514b134407f7d2d507cfab7a938025acb0dae929881"
Sep 14 03:07:40.006767 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.006735    2579 generic.go:356] "Generic (PLEG): container finished" podID="2971ed6e-54f9-46e1-b5b3-b740fcbf3170" containerID="93222743da3f93a2785d451011abe1039a4bd48bf6c1773fd0632f60433090c7" exitCode=0
Sep 14 03:07:40.007290 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.006830    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr" event={"ID":"2971ed6e-54f9-46e1-b5b3-b740fcbf3170","Type":"ContainerDied","Data":"93222743da3f93a2785d451011abe1039a4bd48bf6c1773fd0632f60433090c7"}
Sep 14 03:07:40.007404 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.007383    2579 scope.go:117] "RemoveContainer" containerID="93222743da3f93a2785d451011abe1039a4bd48bf6c1773fd0632f60433090c7"
Sep 14 03:07:40.008770 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.008735    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:07:40.008916 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.008826    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:07:40.008916 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.008873    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:07:40.025115 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.025075    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"]
Sep 14 03:07:40.025356 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.025319    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk" podUID="69683372-d185-4d06-b375-da9936e7d07b" containerName="payload-processing" containerID="cri-o://1417d3a7f96e67bb5acf0dbac53d24939d9282bcd4eb2501901c572cbaca9308" gracePeriod=30
Sep 14 03:07:40.215503 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.215454    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq"]
Sep 14 03:07:40.218986 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.218364    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fb76f4c2-8l5lq"]
Sep 14 03:07:40.300504 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.300474    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"
Sep 14 03:07:40.346592 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.346494    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"]
Sep 14 03:07:40.394737 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.394705    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69683372-d185-4d06-b375-da9936e7d07b-plugins-config-volume\") pod \"69683372-d185-4d06-b375-da9936e7d07b\" (UID: \"69683372-d185-4d06-b375-da9936e7d07b\") "
Sep 14 03:07:40.394895 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.394752    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnvgk\" (UniqueName: \"kubernetes.io/projected/69683372-d185-4d06-b375-da9936e7d07b-kube-api-access-lnvgk\") pod \"69683372-d185-4d06-b375-da9936e7d07b\" (UID: \"69683372-d185-4d06-b375-da9936e7d07b\") "
Sep 14 03:07:40.395435 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.395400    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69683372-d185-4d06-b375-da9936e7d07b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "69683372-d185-4d06-b375-da9936e7d07b" (UID: "69683372-d185-4d06-b375-da9936e7d07b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:07:40.396941 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.396913    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69683372-d185-4d06-b375-da9936e7d07b-kube-api-access-lnvgk" (OuterVolumeSpecName: "kube-api-access-lnvgk") pod "69683372-d185-4d06-b375-da9936e7d07b" (UID: "69683372-d185-4d06-b375-da9936e7d07b"). InnerVolumeSpecName "kube-api-access-lnvgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:07:40.495517 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.495479    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69683372-d185-4d06-b375-da9936e7d07b-plugins-config-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:40.495517 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:40.495511    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnvgk\" (UniqueName: \"kubernetes.io/projected/69683372-d185-4d06-b375-da9936e7d07b-kube-api-access-lnvgk\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:41.015050 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:41.015009    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr" event={"ID":"2971ed6e-54f9-46e1-b5b3-b740fcbf3170","Type":"ContainerStarted","Data":"248e309bc7785850e8cb990e974318b9e2d3c30e1fb580ad03b1092cb5a10f5c"}
Sep 14 03:07:41.015552 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:41.015056    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr" podUID="2971ed6e-54f9-46e1-b5b3-b740fcbf3170" containerName="istio-proxy" containerID="cri-o://248e309bc7785850e8cb990e974318b9e2d3c30e1fb580ad03b1092cb5a10f5c" gracePeriod=30
Sep 14 03:07:41.016664 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:41.016635    2579 generic.go:356] "Generic (PLEG): container finished" podID="69683372-d185-4d06-b375-da9936e7d07b" containerID="1417d3a7f96e67bb5acf0dbac53d24939d9282bcd4eb2501901c572cbaca9308" exitCode=0
Sep 14 03:07:41.016756 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:41.016703    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk" event={"ID":"69683372-d185-4d06-b375-da9936e7d07b","Type":"ContainerDied","Data":"1417d3a7f96e67bb5acf0dbac53d24939d9282bcd4eb2501901c572cbaca9308"}
Sep 14 03:07:41.016756 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:41.016737    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"
Sep 14 03:07:41.016756 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:41.016748    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk" event={"ID":"69683372-d185-4d06-b375-da9936e7d07b","Type":"ContainerDied","Data":"2b32d9f077cd575da61dc77848e539306292ec7351bf9f8bbf07e7c0419cc61f"}
Sep 14 03:07:41.016852 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:41.016773    2579 scope.go:117] "RemoveContainer" containerID="1417d3a7f96e67bb5acf0dbac53d24939d9282bcd4eb2501901c572cbaca9308"
Sep 14 03:07:41.026399 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:41.026382    2579 scope.go:117] "RemoveContainer" containerID="1417d3a7f96e67bb5acf0dbac53d24939d9282bcd4eb2501901c572cbaca9308"
Sep 14 03:07:41.026657 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:07:41.026639    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1417d3a7f96e67bb5acf0dbac53d24939d9282bcd4eb2501901c572cbaca9308\": container with ID starting with 1417d3a7f96e67bb5acf0dbac53d24939d9282bcd4eb2501901c572cbaca9308 not found: ID does not exist" containerID="1417d3a7f96e67bb5acf0dbac53d24939d9282bcd4eb2501901c572cbaca9308"
Sep 14 03:07:41.026762 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:41.026666    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1417d3a7f96e67bb5acf0dbac53d24939d9282bcd4eb2501901c572cbaca9308"} err="failed to get container status \"1417d3a7f96e67bb5acf0dbac53d24939d9282bcd4eb2501901c572cbaca9308\": rpc error: code = NotFound desc = could not find container \"1417d3a7f96e67bb5acf0dbac53d24939d9282bcd4eb2501901c572cbaca9308\": container with ID starting with 1417d3a7f96e67bb5acf0dbac53d24939d9282bcd4eb2501901c572cbaca9308 not found: ID does not exist"
Sep 14 03:07:41.066950 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:41.066921    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"]
Sep 14 03:07:41.085757 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:41.085721    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fb76f4c2-6ddbc8bbd5-bm8lk"]
Sep 14 03:07:41.744457 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:41.744428    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69683372-d185-4d06-b375-da9936e7d07b" path="/var/lib/kubelet/pods/69683372-d185-4d06-b375-da9936e7d07b/volumes"
Sep 14 03:07:41.744761 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:41.744749    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d902dcec-45c9-4ea8-ae12-ed6750ce2e9e" path="/var/lib/kubelet/pods/d902dcec-45c9-4ea8-ae12-ed6750ce2e9e/volumes"
Sep 14 03:07:44.868476 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:44.868433    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:46.264241 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.264216    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:46.344009 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.343946    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-token\") pod \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") "
Sep 14 03:07:46.344009 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.343980    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zplf\" (UniqueName: \"kubernetes.io/projected/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-kube-api-access-2zplf\") pod \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") "
Sep 14 03:07:46.344158 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.344015    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-workload-socket\") pod \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") "
Sep 14 03:07:46.344158 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.344053    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istiod-ca-cert\") pod \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") "
Sep 14 03:07:46.344158 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.344105    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-workload-certs\") pod \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") "
Sep 14 03:07:46.344158 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.344135    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-credential-socket\") pod \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") "
Sep 14 03:07:46.344158 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.344154    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-podinfo\") pod \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") "
Sep 14 03:07:46.344408 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.344233    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-envoy\") pod \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") "
Sep 14 03:07:46.344408 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.344253    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-data\") pod \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\" (UID: \"2971ed6e-54f9-46e1-b5b3-b740fcbf3170\") "
Sep 14 03:07:46.344408 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.344359    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2971ed6e-54f9-46e1-b5b3-b740fcbf3170" (UID: "2971ed6e-54f9-46e1-b5b3-b740fcbf3170"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:46.344558 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.344520    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2971ed6e-54f9-46e1-b5b3-b740fcbf3170" (UID: "2971ed6e-54f9-46e1-b5b3-b740fcbf3170"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:46.344690 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.344558    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2971ed6e-54f9-46e1-b5b3-b740fcbf3170" (UID: "2971ed6e-54f9-46e1-b5b3-b740fcbf3170"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:07:46.344690 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.344595    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-data" (OuterVolumeSpecName: "istio-data") pod "2971ed6e-54f9-46e1-b5b3-b740fcbf3170" (UID: "2971ed6e-54f9-46e1-b5b3-b740fcbf3170"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:46.344690 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.344580    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:46.344945 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.344921    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2971ed6e-54f9-46e1-b5b3-b740fcbf3170" (UID: "2971ed6e-54f9-46e1-b5b3-b740fcbf3170"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:46.346264 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.346241    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-token" (OuterVolumeSpecName: "istio-token") pod "2971ed6e-54f9-46e1-b5b3-b740fcbf3170" (UID: "2971ed6e-54f9-46e1-b5b3-b740fcbf3170"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:07:46.346456 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.346429    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2971ed6e-54f9-46e1-b5b3-b740fcbf3170" (UID: "2971ed6e-54f9-46e1-b5b3-b740fcbf3170"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:07:46.346532 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.346505    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-kube-api-access-2zplf" (OuterVolumeSpecName: "kube-api-access-2zplf") pod "2971ed6e-54f9-46e1-b5b3-b740fcbf3170" (UID: "2971ed6e-54f9-46e1-b5b3-b740fcbf3170"). InnerVolumeSpecName "kube-api-access-2zplf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:07:46.346635 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.346618    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2971ed6e-54f9-46e1-b5b3-b740fcbf3170" (UID: "2971ed6e-54f9-46e1-b5b3-b740fcbf3170"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:07:46.446010 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.445974    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:46.446010 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.446010    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:46.446165 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.446024    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:46.446165 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.446038    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:46.446165 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.446048    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:46.446165 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.446061    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zplf\" (UniqueName: \"kubernetes.io/projected/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-kube-api-access-2zplf\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:46.446165 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.446073    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:46.446165 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:46.446086    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2971ed6e-54f9-46e1-b5b3-b740fcbf3170-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:07:47.056724 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:47.056685    2579 generic.go:356] "Generic (PLEG): container finished" podID="2971ed6e-54f9-46e1-b5b3-b740fcbf3170" containerID="248e309bc7785850e8cb990e974318b9e2d3c30e1fb580ad03b1092cb5a10f5c" exitCode=0
Sep 14 03:07:47.056925 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:47.056742    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"
Sep 14 03:07:47.056925 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:47.056740    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr" event={"ID":"2971ed6e-54f9-46e1-b5b3-b740fcbf3170","Type":"ContainerDied","Data":"248e309bc7785850e8cb990e974318b9e2d3c30e1fb580ad03b1092cb5a10f5c"}
Sep 14 03:07:47.056925 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:47.056864    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr" event={"ID":"2971ed6e-54f9-46e1-b5b3-b740fcbf3170","Type":"ContainerDied","Data":"7e2a6392974844b346d3e92c0ccee3fc0d58bd051524148a1c1ab90261d5b790"}
Sep 14 03:07:47.056925 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:47.056897    2579 scope.go:117] "RemoveContainer" containerID="248e309bc7785850e8cb990e974318b9e2d3c30e1fb580ad03b1092cb5a10f5c"
Sep 14 03:07:47.066398 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:47.066378    2579 scope.go:117] "RemoveContainer" containerID="93222743da3f93a2785d451011abe1039a4bd48bf6c1773fd0632f60433090c7"
Sep 14 03:07:47.074550 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:47.074532    2579 scope.go:117] "RemoveContainer" containerID="248e309bc7785850e8cb990e974318b9e2d3c30e1fb580ad03b1092cb5a10f5c"
Sep 14 03:07:47.074794 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:07:47.074775    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"248e309bc7785850e8cb990e974318b9e2d3c30e1fb580ad03b1092cb5a10f5c\": container with ID starting with 248e309bc7785850e8cb990e974318b9e2d3c30e1fb580ad03b1092cb5a10f5c not found: ID does not exist" containerID="248e309bc7785850e8cb990e974318b9e2d3c30e1fb580ad03b1092cb5a10f5c"
Sep 14 03:07:47.074858 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:47.074801    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248e309bc7785850e8cb990e974318b9e2d3c30e1fb580ad03b1092cb5a10f5c"} err="failed to get container status \"248e309bc7785850e8cb990e974318b9e2d3c30e1fb580ad03b1092cb5a10f5c\": rpc error: code = NotFound desc = could not find container \"248e309bc7785850e8cb990e974318b9e2d3c30e1fb580ad03b1092cb5a10f5c\": container with ID starting with 248e309bc7785850e8cb990e974318b9e2d3c30e1fb580ad03b1092cb5a10f5c not found: ID does not exist"
Sep 14 03:07:47.074858 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:47.074816    2579 scope.go:117] "RemoveContainer" containerID="93222743da3f93a2785d451011abe1039a4bd48bf6c1773fd0632f60433090c7"
Sep 14 03:07:47.075071 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:07:47.075052    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93222743da3f93a2785d451011abe1039a4bd48bf6c1773fd0632f60433090c7\": container with ID starting with 93222743da3f93a2785d451011abe1039a4bd48bf6c1773fd0632f60433090c7 not found: ID does not exist" containerID="93222743da3f93a2785d451011abe1039a4bd48bf6c1773fd0632f60433090c7"
Sep 14 03:07:47.075111 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:47.075077    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93222743da3f93a2785d451011abe1039a4bd48bf6c1773fd0632f60433090c7"} err="failed to get container status \"93222743da3f93a2785d451011abe1039a4bd48bf6c1773fd0632f60433090c7\": rpc error: code = NotFound desc = could not find container \"93222743da3f93a2785d451011abe1039a4bd48bf6c1773fd0632f60433090c7\": container with ID starting with 93222743da3f93a2785d451011abe1039a4bd48bf6c1773fd0632f60433090c7 not found: ID does not exist"
Sep 14 03:07:47.086447 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:47.086427    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"]
Sep 14 03:07:47.092366 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:47.092342    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-fb76f4c2-openshift-default-59b4dc47c6-d2gtr"]
Sep 14 03:07:47.745328 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:07:47.745288    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2971ed6e-54f9-46e1-b5b3-b740fcbf3170" path="/var/lib/kubelet/pods/2971ed6e-54f9-46e1-b5b3-b740fcbf3170/volumes"
Sep 14 03:08:02.178948 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.178918    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"]
Sep 14 03:08:02.179353 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.179335    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d902dcec-45c9-4ea8-ae12-ed6750ce2e9e" containerName="revoke-keys"
Sep 14 03:08:02.179353 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.179350    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d902dcec-45c9-4ea8-ae12-ed6750ce2e9e" containerName="revoke-keys"
Sep 14 03:08:02.179435 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.179365    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2971ed6e-54f9-46e1-b5b3-b740fcbf3170" containerName="istio-proxy"
Sep 14 03:08:02.179435 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.179371    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2971ed6e-54f9-46e1-b5b3-b740fcbf3170" containerName="istio-proxy"
Sep 14 03:08:02.179435 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.179385    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69683372-d185-4d06-b375-da9936e7d07b" containerName="payload-processing"
Sep 14 03:08:02.179435 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.179390    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="69683372-d185-4d06-b375-da9936e7d07b" containerName="payload-processing"
Sep 14 03:08:02.179560 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.179440    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2971ed6e-54f9-46e1-b5b3-b740fcbf3170" containerName="istio-proxy"
Sep 14 03:08:02.179560 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.179447    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="69683372-d185-4d06-b375-da9936e7d07b" containerName="payload-processing"
Sep 14 03:08:02.179560 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.179456    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d902dcec-45c9-4ea8-ae12-ed6750ce2e9e" containerName="revoke-keys"
Sep 14 03:08:02.183796 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.183774    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.189108 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.189084    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-9c9f1e\"/\"kube-root-ca.crt\""
Sep 14 03:08:02.189681 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.189652    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-9c9f1e\"/\"openshift-service-ca.crt\""
Sep 14 03:08:02.189762 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.189662    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-9c9f1e\"/\"premium-w4-9c9f1e-kserve-self-signed-certs\""
Sep 14 03:08:02.190535 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.190376    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-9c9f1e\"/\"default-dockercfg-5qjfv\""
Sep 14 03:08:02.201664 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.201638    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"]
Sep 14 03:08:02.285045 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.285014    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-tmp-dir\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.285045 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.285051    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d96d89d-13aa-4257-936b-eba41890eeb2-tls-certs\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.285297 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.285082    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-model-cache\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.285297 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.285113    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-home\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.285297 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.285208    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-dshm\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.285297 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.285228    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfjbr\" (UniqueName: \"kubernetes.io/projected/7d96d89d-13aa-4257-936b-eba41890eeb2-kube-api-access-vfjbr\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.386558 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.386524    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-tmp-dir\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.386558 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.386560    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d96d89d-13aa-4257-936b-eba41890eeb2-tls-certs\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.386769 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.386593    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-model-cache\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.386769 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.386613    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-home\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.386769 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.386669    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-dshm\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.386769 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.386687    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfjbr\" (UniqueName: \"kubernetes.io/projected/7d96d89d-13aa-4257-936b-eba41890eeb2-kube-api-access-vfjbr\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.387024 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.386997    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-model-cache\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.387088 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.387018    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-tmp-dir\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.387088 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.387060    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-home\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.389002 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.388985    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-dshm\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.389154 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.389138    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d96d89d-13aa-4257-936b-eba41890eeb2-tls-certs\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.396250 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.396226    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfjbr\" (UniqueName: \"kubernetes.io/projected/7d96d89d-13aa-4257-936b-eba41890eeb2-kube-api-access-vfjbr\") pod \"premium-w4-9c9f1e-kserve-557d986567-2jdhx\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") " pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.493557 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.493524    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:02.766620 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:02.766593    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"]
Sep 14 03:08:02.768230 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:08:02.768203    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d96d89d_13aa_4257_936b_eba41890eeb2.slice/crio-b11ca64040281ea1b393222db192e48e7e9afe9b0c170dea430cf0f01890396b WatchSource:0}: Error finding container b11ca64040281ea1b393222db192e48e7e9afe9b0c170dea430cf0f01890396b: Status 404 returned error can't find the container with id b11ca64040281ea1b393222db192e48e7e9afe9b0c170dea430cf0f01890396b
Sep 14 03:08:03.147872 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:03.147789    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx" event={"ID":"7d96d89d-13aa-4257-936b-eba41890eeb2","Type":"ContainerStarted","Data":"e306a381bfac55ade345e09cd0b5646772a8d5fcdcb907af9ab60fdedd57c63f"}
Sep 14 03:08:03.147872 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:03.147826    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx" event={"ID":"7d96d89d-13aa-4257-936b-eba41890eeb2","Type":"ContainerStarted","Data":"b11ca64040281ea1b393222db192e48e7e9afe9b0c170dea430cf0f01890396b"}
Sep 14 03:08:03.168555 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:03.168513    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx" podStartSLOduration=1.168498853 podStartE2EDuration="1.168498853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:08:03.167103856 +0000 UTC m=+945.953165765" watchObservedRunningTime="2026-09-14 03:08:03.168498853 +0000 UTC m=+945.954560766"
Sep 14 03:08:06.688140 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.688097    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"]
Sep 14 03:08:06.688581 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.688537    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2971ed6e-54f9-46e1-b5b3-b740fcbf3170" containerName="istio-proxy"
Sep 14 03:08:06.688581 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.688550    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2971ed6e-54f9-46e1-b5b3-b740fcbf3170" containerName="istio-proxy"
Sep 14 03:08:06.688699 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.688617    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2971ed6e-54f9-46e1-b5b3-b740fcbf3170" containerName="istio-proxy"
Sep 14 03:08:06.698194 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.692129    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.698194 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.695843    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-8364a0-openshift-default-dockercfg-rwj5q\""
Sep 14 03:08:06.708509 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.708482    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"]
Sep 14 03:08:06.831185 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.831137    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-credential-socket\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.831355 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.831201    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-envoy\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.831355 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.831280    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-token\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.831355 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.831302    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-podinfo\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.831355 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.831320    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istiod-ca-cert\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.831355 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.831347    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7nr8\" (UniqueName: \"kubernetes.io/projected/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-kube-api-access-g7nr8\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.831581 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.831394    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-workload-socket\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.831581 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.831410    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-data\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.831581 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.831426    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-workload-certs\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.932047 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.932015    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-token\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.932281 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.932058    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-podinfo\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.932281 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.932088    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istiod-ca-cert\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.932281 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.932120    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7nr8\" (UniqueName: \"kubernetes.io/projected/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-kube-api-access-g7nr8\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.932281 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.932160    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-workload-socket\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.932281 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.932218    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-data\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.932281 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.932245    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-workload-certs\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.932281 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.932273    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-credential-socket\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.932639 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.932314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-envoy\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.932703 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.932672    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-data\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.932771 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.932703    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-workload-certs\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.932863 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.932832    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-credential-socket\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.932987 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.932939    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-workload-socket\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.933101 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.933078    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istiod-ca-cert\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.934650 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.934628    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-envoy\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.934780 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.934762    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-podinfo\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.940631 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.940581    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-token\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:06.941004 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:06.940982    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7nr8\" (UniqueName: \"kubernetes.io/projected/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-kube-api-access-g7nr8\") pod \"e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") " pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:07.010838 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:07.010804    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:07.307318 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:07.307293    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"]
Sep 14 03:08:07.309785 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:08:07.309755    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0478c9a_11f7_4bec_a23d_3a439f3c5d2a.slice/crio-4a0fe2fd0b69e4379f41e098e38ebfa34672d05f15bfe919a09b7d37784b45df WatchSource:0}: Error finding container 4a0fe2fd0b69e4379f41e098e38ebfa34672d05f15bfe919a09b7d37784b45df: Status 404 returned error can't find the container with id 4a0fe2fd0b69e4379f41e098e38ebfa34672d05f15bfe919a09b7d37784b45df
Sep 14 03:08:07.312029 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:07.311997    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:08:07.312126 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:07.312065    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:08:07.312126 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:07.312091    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:08:08.173915 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:08.173870    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg" event={"ID":"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a","Type":"ContainerStarted","Data":"c76e1ff08fccb7696d064c303eba92f1ca68dfb98802c57290394828af60ed95"}
Sep 14 03:08:08.174317 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:08.173938    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg" event={"ID":"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a","Type":"ContainerStarted","Data":"4a0fe2fd0b69e4379f41e098e38ebfa34672d05f15bfe919a09b7d37784b45df"}
Sep 14 03:08:08.197456 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:08.197406    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg" podStartSLOduration=2.197391416 podStartE2EDuration="2.197391416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:08:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:08:08.194673896 +0000 UTC m=+950.980735817" watchObservedRunningTime="2026-09-14 03:08:08.197391416 +0000 UTC m=+950.983453329"
Sep 14 03:08:09.011747 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:09.011716    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:09.016396 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:09.016371    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:09.178678 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:09.178649    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:09.179478 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:09.179458    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:12.494055 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:12.494023    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:12.494536 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:12.494080    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:12.506621 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:12.506596    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:13.213764 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.213735    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:08:13.435347 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.435319    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"]
Sep 14 03:08:13.438941 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.438920    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:08:13.445168 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.445144    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-8364a0-serving-cert\""
Sep 14 03:08:13.445878 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.445852    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 03:08:13.445878 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.445870    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jf78h\""
Sep 14 03:08:13.446363 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.446039    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 03:08:13.446363 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.446148    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:08:13.452860 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.452836    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"]
Sep 14 03:08:13.487839 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.487779    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/479c9432-13a6-4736-af25-40eae7184415-maas-api-tls\") pod \"maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm\" (UID: \"479c9432-13a6-4736-af25-40eae7184415\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:08:13.487839 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.487825    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/479c9432-13a6-4736-af25-40eae7184415-openshift-service-ca\") pod \"maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm\" (UID: \"479c9432-13a6-4736-af25-40eae7184415\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:08:13.488012 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.487895    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxmgc\" (UniqueName: \"kubernetes.io/projected/479c9432-13a6-4736-af25-40eae7184415-kube-api-access-dxmgc\") pod \"maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm\" (UID: \"479c9432-13a6-4736-af25-40eae7184415\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:08:13.488012 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.487991    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/479c9432-13a6-4736-af25-40eae7184415-metrics-certs\") pod \"maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm\" (UID: \"479c9432-13a6-4736-af25-40eae7184415\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:08:13.588873 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.588847    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/479c9432-13a6-4736-af25-40eae7184415-metrics-certs\") pod \"maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm\" (UID: \"479c9432-13a6-4736-af25-40eae7184415\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:08:13.589325 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.588897    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/479c9432-13a6-4736-af25-40eae7184415-maas-api-tls\") pod \"maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm\" (UID: \"479c9432-13a6-4736-af25-40eae7184415\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:08:13.589325 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.588932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/479c9432-13a6-4736-af25-40eae7184415-openshift-service-ca\") pod \"maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm\" (UID: \"479c9432-13a6-4736-af25-40eae7184415\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:08:13.589325 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.588961    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxmgc\" (UniqueName: \"kubernetes.io/projected/479c9432-13a6-4736-af25-40eae7184415-kube-api-access-dxmgc\") pod \"maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm\" (UID: \"479c9432-13a6-4736-af25-40eae7184415\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:08:13.589713 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.589693    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/479c9432-13a6-4736-af25-40eae7184415-openshift-service-ca\") pod \"maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm\" (UID: \"479c9432-13a6-4736-af25-40eae7184415\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:08:13.591452 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.591437    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/479c9432-13a6-4736-af25-40eae7184415-metrics-certs\") pod \"maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm\" (UID: \"479c9432-13a6-4736-af25-40eae7184415\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:08:13.591615 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.591592    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/479c9432-13a6-4736-af25-40eae7184415-maas-api-tls\") pod \"maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm\" (UID: \"479c9432-13a6-4736-af25-40eae7184415\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:08:13.599816 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.599784    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxmgc\" (UniqueName: \"kubernetes.io/projected/479c9432-13a6-4736-af25-40eae7184415-kube-api-access-dxmgc\") pod \"maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm\" (UID: \"479c9432-13a6-4736-af25-40eae7184415\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:08:13.749830 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:13.749807    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:08:14.033290 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:14.033265    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"]
Sep 14 03:08:14.035064 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:08:14.035038    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod479c9432_13a6_4736_af25_40eae7184415.slice/crio-489e6af918b35aac66f0302762a1f5bc6611ca80b52d80ec0ebe135978a9c450 WatchSource:0}: Error finding container 489e6af918b35aac66f0302762a1f5bc6611ca80b52d80ec0ebe135978a9c450: Status 404 returned error can't find the container with id 489e6af918b35aac66f0302762a1f5bc6611ca80b52d80ec0ebe135978a9c450
Sep 14 03:08:14.036540 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:14.036518    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 03:08:14.213806 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:14.213773    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm" event={"ID":"479c9432-13a6-4736-af25-40eae7184415","Type":"ContainerStarted","Data":"489e6af918b35aac66f0302762a1f5bc6611ca80b52d80ec0ebe135978a9c450"}
Sep 14 03:08:15.220973 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:15.220931    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm" event={"ID":"479c9432-13a6-4736-af25-40eae7184415","Type":"ContainerStarted","Data":"ac4f08bd84979cca13c87856cb8f82a5acfa34535d2a21e17e34b75c0bf61c6e"}
Sep 14 03:08:15.221442 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:15.221064    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:08:15.241289 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:15.241240    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm" podStartSLOduration=1.790142227 podStartE2EDuration="2.241225177s" totalImagesPullingTime="451.08295ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:08:15.240581862 +0000 UTC m=+958.026643775" watchObservedRunningTime="2026-09-14 03:08:15.241225177 +0000 UTC m=+958.027287087"
Sep 14 03:08:21.231234 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:21.231200    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:08:39.180076 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:39.180040    2579 patch_prober.go:28] interesting pod/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 14 03:08:39.180472 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:39.180106    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg" podUID="d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 14 03:08:39.353274 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:39.353162    2579 generic.go:356] "Generic (PLEG): container finished" podID="d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" containerID="c76e1ff08fccb7696d064c303eba92f1ca68dfb98802c57290394828af60ed95" exitCode=0
Sep 14 03:08:39.353274 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:39.353213    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg" event={"ID":"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a","Type":"ContainerDied","Data":"c76e1ff08fccb7696d064c303eba92f1ca68dfb98802c57290394828af60ed95"}
Sep 14 03:08:39.353607 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:39.353593    2579 scope.go:117] "RemoveContainer" containerID="c76e1ff08fccb7696d064c303eba92f1ca68dfb98802c57290394828af60ed95"
Sep 14 03:08:39.354713 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:39.354686    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:08:39.354774 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:39.354754    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:08:39.354810 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:39.354785    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:08:40.360406 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:40.360365    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg" event={"ID":"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a","Type":"ContainerStarted","Data":"13525c96706097fe8406140fa43d3986b64e1f4090ed25b982962cd532266ce8"}
Sep 14 03:08:41.011417 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:41.011383    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:41.011800 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:41.011705    2579 patch_prober.go:28] interesting pod/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 14 03:08:41.011995 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:41.011970    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg" podUID="d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 14 03:08:42.011549 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:42.011509    2579 patch_prober.go:28] interesting pod/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 14 03:08:42.012199 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:42.012138    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg" podUID="d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 14 03:08:43.011962 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:43.011920    2579 patch_prober.go:28] interesting pod/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 14 03:08:43.012574 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:43.012543    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg" podUID="d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 14 03:08:44.011899 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:44.011862    2579 patch_prober.go:28] interesting pod/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 14 03:08:44.012228 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:44.012199    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg" podUID="d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 14 03:08:45.012595 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:45.012101    2579 patch_prober.go:28] interesting pod/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 14 03:08:45.012595 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:45.012197    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg" podUID="d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 14 03:08:46.016066 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:46.016035    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:46.016512 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:46.016402    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:08:46.017033 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:08:46.017017    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:09:18.553748 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.553714    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"]
Sep 14 03:09:18.556803 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.556781    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.561046 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.561024    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-446c5fbd-openshift-default-dockercfg-d8d8z\""
Sep 14 03:09:18.573011 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.572983    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"]
Sep 14 03:09:18.668334 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.668294    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-workload-certs\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.668527 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.668344    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43c18a7d-9f1c-4bf3-b813-a13098812df7-istiod-ca-cert\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.668527 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.668385    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-envoy\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.668527 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.668424    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzfbr\" (UniqueName: \"kubernetes.io/projected/43c18a7d-9f1c-4bf3-b813-a13098812df7-kube-api-access-vzfbr\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.668527 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.668462    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-token\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.668796 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.668546    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-credential-socket\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.668796 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.668602    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-podinfo\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.668796 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.668640    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-data\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.668796 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.668677    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-workload-socket\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.769462 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.769426    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-workload-certs\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.769462 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.769465    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43c18a7d-9f1c-4bf3-b813-a13098812df7-istiod-ca-cert\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.769707 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.769496    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-envoy\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.769707 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.769521    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzfbr\" (UniqueName: \"kubernetes.io/projected/43c18a7d-9f1c-4bf3-b813-a13098812df7-kube-api-access-vzfbr\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.769707 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.769555    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-token\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.769707 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.769583    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-credential-socket\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.769896 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.769802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-podinfo\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.769896 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.769869    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-data\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.770004 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.769912    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-workload-socket\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.770004 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.769990    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-credential-socket\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.770138 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.770120    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-workload-certs\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.770404 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.770242    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-data\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.770404 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.770290    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-workload-socket\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.770404 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.770305    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43c18a7d-9f1c-4bf3-b813-a13098812df7-istiod-ca-cert\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.771898 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.771878    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-envoy\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.772291 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.772272    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-podinfo\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.778793 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.778772    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-token\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.778872 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.778776    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzfbr\" (UniqueName: \"kubernetes.io/projected/43c18a7d-9f1c-4bf3-b813-a13098812df7-kube-api-access-vzfbr\") pod \"e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") " pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:18.869509 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:18.869417    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:19.286147 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:09:19.286116    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c18a7d_9f1c_4bf3_b813_a13098812df7.slice/crio-7d8cafbcde5482c33423ef0e5e5a8f2bdd2ab8b0462b3dbf136aeadd420a0d41 WatchSource:0}: Error finding container 7d8cafbcde5482c33423ef0e5e5a8f2bdd2ab8b0462b3dbf136aeadd420a0d41: Status 404 returned error can't find the container with id 7d8cafbcde5482c33423ef0e5e5a8f2bdd2ab8b0462b3dbf136aeadd420a0d41
Sep 14 03:09:19.288619 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:19.288587    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:09:19.288746 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:19.288674    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:09:19.288746 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:19.288716    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:09:19.396021 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:19.395988    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"]
Sep 14 03:09:19.576422 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:19.576338    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h" event={"ID":"43c18a7d-9f1c-4bf3-b813-a13098812df7","Type":"ContainerStarted","Data":"df5ebe4e36030c499792b1bd0091c757e59b34d2b89cf7392dd554ae43e74683"}
Sep 14 03:09:19.576422 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:19.576404    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h" event={"ID":"43c18a7d-9f1c-4bf3-b813-a13098812df7","Type":"ContainerStarted","Data":"7d8cafbcde5482c33423ef0e5e5a8f2bdd2ab8b0462b3dbf136aeadd420a0d41"}
Sep 14 03:09:19.604598 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:19.604546    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h" podStartSLOduration=1.604525676 podStartE2EDuration="1.604525676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:09:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:09:19.602282155 +0000 UTC m=+1022.388344079" watchObservedRunningTime="2026-09-14 03:09:19.604525676 +0000 UTC m=+1022.390587591"
Sep 14 03:09:19.870367 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:19.870286    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:20.875136 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:20.875102    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:21.592701 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:21.592662    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:21.593902 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:21.593880    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:29.215257 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.215221    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"]
Sep 14 03:09:29.218463 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.218439    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:29.221306 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.221284    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-446c5fbd-serving-cert\""
Sep 14 03:09:29.231074 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.231049    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"]
Sep 14 03:09:29.274382 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.274349    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fftmt\" (UniqueName: \"kubernetes.io/projected/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-kube-api-access-fftmt\") pod \"maas-api-e2e-mt-446c5fbd-747bc59999-45lcj\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:29.274505 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.274388    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-openshift-service-ca\") pod \"maas-api-e2e-mt-446c5fbd-747bc59999-45lcj\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:29.274505 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.274424    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-maas-api-tls\") pod \"maas-api-e2e-mt-446c5fbd-747bc59999-45lcj\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:29.274505 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.274448    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-metrics-certs\") pod \"maas-api-e2e-mt-446c5fbd-747bc59999-45lcj\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:29.375306 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.375274    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fftmt\" (UniqueName: \"kubernetes.io/projected/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-kube-api-access-fftmt\") pod \"maas-api-e2e-mt-446c5fbd-747bc59999-45lcj\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:29.375306 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.375308    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-openshift-service-ca\") pod \"maas-api-e2e-mt-446c5fbd-747bc59999-45lcj\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:29.375599 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.375334    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-maas-api-tls\") pod \"maas-api-e2e-mt-446c5fbd-747bc59999-45lcj\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:29.375599 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.375353    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-metrics-certs\") pod \"maas-api-e2e-mt-446c5fbd-747bc59999-45lcj\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:29.375599 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:09:29.375475    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-serving-cert: secret "maas-api-e2e-mt-446c5fbd-serving-cert" not found
Sep 14 03:09:29.375599 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:09:29.375562    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-maas-api-tls podName:c95bb3c7-55b9-4241-a4e7-a34af6b4c53c nodeName:}" failed. No retries permitted until 2026-09-14 03:09:29.875539839 +0000 UTC m=+1032.661601733 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-maas-api-tls") pod "maas-api-e2e-mt-446c5fbd-747bc59999-45lcj" (UID: "c95bb3c7-55b9-4241-a4e7-a34af6b4c53c") : secret "maas-api-e2e-mt-446c5fbd-serving-cert" not found
Sep 14 03:09:29.376026 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.376003    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-openshift-service-ca\") pod \"maas-api-e2e-mt-446c5fbd-747bc59999-45lcj\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:29.378051 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.378022    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-metrics-certs\") pod \"maas-api-e2e-mt-446c5fbd-747bc59999-45lcj\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:29.385259 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.385234    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fftmt\" (UniqueName: \"kubernetes.io/projected/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-kube-api-access-fftmt\") pod \"maas-api-e2e-mt-446c5fbd-747bc59999-45lcj\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:29.882279 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.882246    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-maas-api-tls\") pod \"maas-api-e2e-mt-446c5fbd-747bc59999-45lcj\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:29.884980 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:29.884953    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-maas-api-tls\") pod \"maas-api-e2e-mt-446c5fbd-747bc59999-45lcj\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:30.131154 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:30.131123    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:30.404893 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:30.404867    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"]
Sep 14 03:09:30.407113 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:09:30.407083    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc95bb3c7_55b9_4241_a4e7_a34af6b4c53c.slice/crio-694ddcd06043946cc89c00a1ee1fd99bdec23dbca68206b36578c73ab93a4944 WatchSource:0}: Error finding container 694ddcd06043946cc89c00a1ee1fd99bdec23dbca68206b36578c73ab93a4944: Status 404 returned error can't find the container with id 694ddcd06043946cc89c00a1ee1fd99bdec23dbca68206b36578c73ab93a4944
Sep 14 03:09:30.640519 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:30.640484    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj" event={"ID":"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c","Type":"ContainerStarted","Data":"694ddcd06043946cc89c00a1ee1fd99bdec23dbca68206b36578c73ab93a4944"}
Sep 14 03:09:31.646796 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:31.646758    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj" event={"ID":"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c","Type":"ContainerStarted","Data":"5ba4943975b6ec8d8fc4d759e30a6b03f478d6e17fcbd3873524144f49e2e213"}
Sep 14 03:09:31.647304 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:31.646836    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:31.666540 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:31.666497    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj" podStartSLOduration=2.290203691 podStartE2EDuration="2.666483845s" totalImagesPullingTime="376.280154ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:09:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:09:31.664476449 +0000 UTC m=+1034.450538389" watchObservedRunningTime="2026-09-14 03:09:31.666483845 +0000 UTC m=+1034.452545755"
Sep 14 03:09:37.657576 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:37.657544    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:09:48.751587 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:48.751555    2579 generic.go:356] "Generic (PLEG): container finished" podID="43c18a7d-9f1c-4bf3-b813-a13098812df7" containerID="df5ebe4e36030c499792b1bd0091c757e59b34d2b89cf7392dd554ae43e74683" exitCode=0
Sep 14 03:09:48.752021 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:48.751624    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h" event={"ID":"43c18a7d-9f1c-4bf3-b813-a13098812df7","Type":"ContainerDied","Data":"df5ebe4e36030c499792b1bd0091c757e59b34d2b89cf7392dd554ae43e74683"}
Sep 14 03:09:48.752021 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:48.751964    2579 scope.go:117] "RemoveContainer" containerID="df5ebe4e36030c499792b1bd0091c757e59b34d2b89cf7392dd554ae43e74683"
Sep 14 03:09:48.753098 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:48.753072    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:09:48.753164 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:48.753146    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:09:48.753250 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:48.753199    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:09:49.758439 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:49.758401    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h" event={"ID":"43c18a7d-9f1c-4bf3-b813-a13098812df7","Type":"ContainerStarted","Data":"92b031560ecbe06d2c1caa817f3842d94368b49ba4e41393539d03f590e55521"}
Sep 14 03:09:49.870087 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:49.870045    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:49.871125 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:49.871102    2579 patch_prober.go:28] interesting pod/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 14 03:09:49.871201 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:49.871155    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h" podUID="43c18a7d-9f1c-4bf3-b813-a13098812df7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 14 03:09:50.870683 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:50.870648    2579 patch_prober.go:28] interesting pod/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 14 03:09:50.871075 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:50.870704    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h" podUID="43c18a7d-9f1c-4bf3-b813-a13098812df7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 14 03:09:51.593263 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:51.593221    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:51.870735 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:51.870647    2579 patch_prober.go:28] interesting pod/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 14 03:09:51.870735 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:51.870700    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h" podUID="43c18a7d-9f1c-4bf3-b813-a13098812df7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 14 03:09:52.870595 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:52.870556    2579 patch_prober.go:28] interesting pod/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 14 03:09:52.870769 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:52.870615    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h" podUID="43c18a7d-9f1c-4bf3-b813-a13098812df7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 14 03:09:53.870740 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:53.870703    2579 patch_prober.go:28] interesting pod/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 14 03:09:53.870920 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:53.870771    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h" podUID="43c18a7d-9f1c-4bf3-b813-a13098812df7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 14 03:09:54.870578 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:54.870534    2579 patch_prober.go:28] interesting pod/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 14 03:09:54.870754 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:54.870605    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h" podUID="43c18a7d-9f1c-4bf3-b813-a13098812df7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 14 03:09:55.875000 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:55.874971    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:09:55.882784 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:09:55.882758    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:10:00.952506 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:00.952471    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"]
Sep 14 03:10:00.953147 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:00.952700    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm" podUID="479c9432-13a6-4736-af25-40eae7184415" containerName="maas-api" containerID="cri-o://ac4f08bd84979cca13c87856cb8f82a5acfa34535d2a21e17e34b75c0bf61c6e" gracePeriod=30
Sep 14 03:10:01.212096 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.212033    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:10:01.280143 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.280115    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxmgc\" (UniqueName: \"kubernetes.io/projected/479c9432-13a6-4736-af25-40eae7184415-kube-api-access-dxmgc\") pod \"479c9432-13a6-4736-af25-40eae7184415\" (UID: \"479c9432-13a6-4736-af25-40eae7184415\") "
Sep 14 03:10:01.280439 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.280156    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/479c9432-13a6-4736-af25-40eae7184415-metrics-certs\") pod \"479c9432-13a6-4736-af25-40eae7184415\" (UID: \"479c9432-13a6-4736-af25-40eae7184415\") "
Sep 14 03:10:01.280439 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.280243    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/479c9432-13a6-4736-af25-40eae7184415-maas-api-tls\") pod \"479c9432-13a6-4736-af25-40eae7184415\" (UID: \"479c9432-13a6-4736-af25-40eae7184415\") "
Sep 14 03:10:01.280439 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.280296    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/479c9432-13a6-4736-af25-40eae7184415-openshift-service-ca\") pod \"479c9432-13a6-4736-af25-40eae7184415\" (UID: \"479c9432-13a6-4736-af25-40eae7184415\") "
Sep 14 03:10:01.280755 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.280723    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/479c9432-13a6-4736-af25-40eae7184415-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "479c9432-13a6-4736-af25-40eae7184415" (UID: "479c9432-13a6-4736-af25-40eae7184415"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:10:01.282592 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.282562    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/479c9432-13a6-4736-af25-40eae7184415-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "479c9432-13a6-4736-af25-40eae7184415" (UID: "479c9432-13a6-4736-af25-40eae7184415"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:10:01.282693 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.282617    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/479c9432-13a6-4736-af25-40eae7184415-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "479c9432-13a6-4736-af25-40eae7184415" (UID: "479c9432-13a6-4736-af25-40eae7184415"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:10:01.282693 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.282660    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/479c9432-13a6-4736-af25-40eae7184415-kube-api-access-dxmgc" (OuterVolumeSpecName: "kube-api-access-dxmgc") pod "479c9432-13a6-4736-af25-40eae7184415" (UID: "479c9432-13a6-4736-af25-40eae7184415"). InnerVolumeSpecName "kube-api-access-dxmgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:10:01.382281 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.382241    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/479c9432-13a6-4736-af25-40eae7184415-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:01.382281 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.382285    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxmgc\" (UniqueName: \"kubernetes.io/projected/479c9432-13a6-4736-af25-40eae7184415-kube-api-access-dxmgc\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:01.382546 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.382300    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/479c9432-13a6-4736-af25-40eae7184415-metrics-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:01.382546 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.382313    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/479c9432-13a6-4736-af25-40eae7184415-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:01.823502 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.823466    2579 generic.go:356] "Generic (PLEG): container finished" podID="479c9432-13a6-4736-af25-40eae7184415" containerID="ac4f08bd84979cca13c87856cb8f82a5acfa34535d2a21e17e34b75c0bf61c6e" exitCode=0
Sep 14 03:10:01.823667 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.823516    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"
Sep 14 03:10:01.823667 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.823540    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm" event={"ID":"479c9432-13a6-4736-af25-40eae7184415","Type":"ContainerDied","Data":"ac4f08bd84979cca13c87856cb8f82a5acfa34535d2a21e17e34b75c0bf61c6e"}
Sep 14 03:10:01.823667 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.823598    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm" event={"ID":"479c9432-13a6-4736-af25-40eae7184415","Type":"ContainerDied","Data":"489e6af918b35aac66f0302762a1f5bc6611ca80b52d80ec0ebe135978a9c450"}
Sep 14 03:10:01.823667 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.823624    2579 scope.go:117] "RemoveContainer" containerID="ac4f08bd84979cca13c87856cb8f82a5acfa34535d2a21e17e34b75c0bf61c6e"
Sep 14 03:10:01.832569 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.832550    2579 scope.go:117] "RemoveContainer" containerID="ac4f08bd84979cca13c87856cb8f82a5acfa34535d2a21e17e34b75c0bf61c6e"
Sep 14 03:10:01.832830 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:10:01.832811    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac4f08bd84979cca13c87856cb8f82a5acfa34535d2a21e17e34b75c0bf61c6e\": container with ID starting with ac4f08bd84979cca13c87856cb8f82a5acfa34535d2a21e17e34b75c0bf61c6e not found: ID does not exist" containerID="ac4f08bd84979cca13c87856cb8f82a5acfa34535d2a21e17e34b75c0bf61c6e"
Sep 14 03:10:01.832891 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.832839    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac4f08bd84979cca13c87856cb8f82a5acfa34535d2a21e17e34b75c0bf61c6e"} err="failed to get container status \"ac4f08bd84979cca13c87856cb8f82a5acfa34535d2a21e17e34b75c0bf61c6e\": rpc error: code = NotFound desc = could not find container \"ac4f08bd84979cca13c87856cb8f82a5acfa34535d2a21e17e34b75c0bf61c6e\": container with ID starting with ac4f08bd84979cca13c87856cb8f82a5acfa34535d2a21e17e34b75c0bf61c6e not found: ID does not exist"
Sep 14 03:10:01.870665 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.870640    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"]
Sep 14 03:10:01.884866 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:01.884838    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-8364a0-7ff8767bf8-76nlm"]
Sep 14 03:10:03.748151 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:03.748113    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="479c9432-13a6-4736-af25-40eae7184415" path="/var/lib/kubelet/pods/479c9432-13a6-4736-af25-40eae7184415/volumes"
Sep 14 03:10:08.418726 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:08.418689    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"]
Sep 14 03:10:08.419107 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:08.418898    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg" podUID="d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" containerName="istio-proxy" containerID="cri-o://13525c96706097fe8406140fa43d3986b64e1f4090ed25b982962cd532266ce8" gracePeriod=30
Sep 14 03:10:09.130286 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.130257    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"]
Sep 14 03:10:09.130666 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.130654    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="479c9432-13a6-4736-af25-40eae7184415" containerName="maas-api"
Sep 14 03:10:09.130719 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.130667    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="479c9432-13a6-4736-af25-40eae7184415" containerName="maas-api"
Sep 14 03:10:09.130758 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.130728    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="479c9432-13a6-4736-af25-40eae7184415" containerName="maas-api"
Sep 14 03:10:09.133527 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.133511    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.136196 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.136161    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-f16030-openshift-default-dockercfg-qhg5b\""
Sep 14 03:10:09.145239 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.145207    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"]
Sep 14 03:10:09.259898 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.259864    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-workload-socket\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.260045 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.259907    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-envoy\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.260045 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.259950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istiod-ca-cert\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.260045 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.260022    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-podinfo\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.260249 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.260054    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-credential-socket\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.260249 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.260080    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-token\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.260249 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.260097    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-data\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.260249 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.260116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-workload-certs\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.260249 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.260184    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn8n4\" (UniqueName: \"kubernetes.io/projected/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-kube-api-access-fn8n4\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.361244 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.361211    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-credential-socket\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.361417 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.361255    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-token\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.361417 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.361284    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-data\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.361539 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.361473    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-workload-certs\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.361938 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.361583    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn8n4\" (UniqueName: \"kubernetes.io/projected/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-kube-api-access-fn8n4\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.361938 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.361627    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-data\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.361938 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.361651    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-credential-socket\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.361938 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.361662    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-workload-socket\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.361938 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.361695    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-envoy\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.361938 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.361719    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istiod-ca-cert\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.361938 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.361809    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-podinfo\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.361938 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.361899    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-workload-certs\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.362429 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.362127    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-workload-socket\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.362476 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.362441    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istiod-ca-cert\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.364078 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.364052    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-envoy\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.364637 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.364618    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-podinfo\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.371974 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.371949    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-token\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.372160 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.372137    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn8n4\" (UniqueName: \"kubernetes.io/projected/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-kube-api-access-fn8n4\") pod \"e2e-noten-f16030-openshift-default-699d768b48-gqkcq\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") " pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.448452 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.448384    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:09.732008 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.731984    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"]
Sep 14 03:10:09.734528 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:10:09.734501    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d022749_c77d_43f1_97fe_21f6a0cbfa2b.slice/crio-a939808cb8696021bd96e4f400b9e7d7674c47c52162bfe6b70e090392cc8b29 WatchSource:0}: Error finding container a939808cb8696021bd96e4f400b9e7d7674c47c52162bfe6b70e090392cc8b29: Status 404 returned error can't find the container with id a939808cb8696021bd96e4f400b9e7d7674c47c52162bfe6b70e090392cc8b29
Sep 14 03:10:09.737075 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.737044    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:10:09.737165 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.737108    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:10:09.737165 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.737135    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:10:09.868981 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.868937    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq" event={"ID":"7d022749-c77d-43f1-97fe-21f6a0cbfa2b","Type":"ContainerStarted","Data":"6ac28a3f5ff46fbe0cf78f9bf72bf6d8655c3586852a827d568e721f878f5529"}
Sep 14 03:10:09.869152 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.869004    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq" event={"ID":"7d022749-c77d-43f1-97fe-21f6a0cbfa2b","Type":"ContainerStarted","Data":"a939808cb8696021bd96e4f400b9e7d7674c47c52162bfe6b70e090392cc8b29"}
Sep 14 03:10:09.891941 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:09.891877    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq" podStartSLOduration=0.89185654 podStartE2EDuration="891.85654ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:10:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:10:09.889046326 +0000 UTC m=+1072.675108240" watchObservedRunningTime="2026-09-14 03:10:09.89185654 +0000 UTC m=+1072.677918454"
Sep 14 03:10:10.449434 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:10.449398    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:10.454854 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:10.454829    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:10.874087 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:10.874048    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:10.875035 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:10.875015    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:10:13.669564 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.669544    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:10:13.803966 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.803939    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-envoy\") pod \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") "
Sep 14 03:10:13.804148 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.803985    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7nr8\" (UniqueName: \"kubernetes.io/projected/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-kube-api-access-g7nr8\") pod \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") "
Sep 14 03:10:13.804148 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.804014    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-workload-certs\") pod \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") "
Sep 14 03:10:13.804148 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.804046    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istiod-ca-cert\") pod \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") "
Sep 14 03:10:13.804148 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.804067    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-podinfo\") pod \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") "
Sep 14 03:10:13.804148 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.804086    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-token\") pod \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") "
Sep 14 03:10:13.804148 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.804104    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-workload-socket\") pod \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") "
Sep 14 03:10:13.804508 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.804208    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-data\") pod \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") "
Sep 14 03:10:13.804508 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.804232    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-credential-socket\") pod \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\" (UID: \"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a\") "
Sep 14 03:10:13.804605 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.804488    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" (UID: "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:10:13.804605 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.804568    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-data" (OuterVolumeSpecName: "istio-data") pod "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" (UID: "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:10:13.804760 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.804688    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" (UID: "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:10:13.804760 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.804746    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" (UID: "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:10:13.804979 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.804768    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" (UID: "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:10:13.806436 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.806415    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" (UID: "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:10:13.806595 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.806539    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" (UID: "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:10:13.806862 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.806841    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-kube-api-access-g7nr8" (OuterVolumeSpecName: "kube-api-access-g7nr8") pod "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" (UID: "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a"). InnerVolumeSpecName "kube-api-access-g7nr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:10:13.806862 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.806852    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-token" (OuterVolumeSpecName: "istio-token") pod "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" (UID: "d0478c9a-11f7-4bec-a23d-3a439f3c5d2a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:10:13.891247 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.891217    2579 generic.go:356] "Generic (PLEG): container finished" podID="d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" containerID="13525c96706097fe8406140fa43d3986b64e1f4090ed25b982962cd532266ce8" exitCode=0
Sep 14 03:10:13.891379 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.891254    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg" event={"ID":"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a","Type":"ContainerDied","Data":"13525c96706097fe8406140fa43d3986b64e1f4090ed25b982962cd532266ce8"}
Sep 14 03:10:13.891379 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.891282    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"
Sep 14 03:10:13.891379 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.891312    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg" event={"ID":"d0478c9a-11f7-4bec-a23d-3a439f3c5d2a","Type":"ContainerDied","Data":"4a0fe2fd0b69e4379f41e098e38ebfa34672d05f15bfe919a09b7d37784b45df"}
Sep 14 03:10:13.891379 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.891335    2579 scope.go:117] "RemoveContainer" containerID="13525c96706097fe8406140fa43d3986b64e1f4090ed25b982962cd532266ce8"
Sep 14 03:10:13.901392 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.901372    2579 scope.go:117] "RemoveContainer" containerID="c76e1ff08fccb7696d064c303eba92f1ca68dfb98802c57290394828af60ed95"
Sep 14 03:10:13.905986 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.905960    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:13.906101 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.905991    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:13.906101 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.906004    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:13.906101 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.906016    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:13.906101 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.906028    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7nr8\" (UniqueName: \"kubernetes.io/projected/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-kube-api-access-g7nr8\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:13.906101 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.906048    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:13.906101 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.906059    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:13.906101 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.906070    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:13.906101 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.906081    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:13.909995 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.909981    2579 scope.go:117] "RemoveContainer" containerID="13525c96706097fe8406140fa43d3986b64e1f4090ed25b982962cd532266ce8"
Sep 14 03:10:13.910244 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:10:13.910225    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13525c96706097fe8406140fa43d3986b64e1f4090ed25b982962cd532266ce8\": container with ID starting with 13525c96706097fe8406140fa43d3986b64e1f4090ed25b982962cd532266ce8 not found: ID does not exist" containerID="13525c96706097fe8406140fa43d3986b64e1f4090ed25b982962cd532266ce8"
Sep 14 03:10:13.910292 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.910252    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13525c96706097fe8406140fa43d3986b64e1f4090ed25b982962cd532266ce8"} err="failed to get container status \"13525c96706097fe8406140fa43d3986b64e1f4090ed25b982962cd532266ce8\": rpc error: code = NotFound desc = could not find container \"13525c96706097fe8406140fa43d3986b64e1f4090ed25b982962cd532266ce8\": container with ID starting with 13525c96706097fe8406140fa43d3986b64e1f4090ed25b982962cd532266ce8 not found: ID does not exist"
Sep 14 03:10:13.910292 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.910267    2579 scope.go:117] "RemoveContainer" containerID="c76e1ff08fccb7696d064c303eba92f1ca68dfb98802c57290394828af60ed95"
Sep 14 03:10:13.910510 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:10:13.910496    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c76e1ff08fccb7696d064c303eba92f1ca68dfb98802c57290394828af60ed95\": container with ID starting with c76e1ff08fccb7696d064c303eba92f1ca68dfb98802c57290394828af60ed95 not found: ID does not exist" containerID="c76e1ff08fccb7696d064c303eba92f1ca68dfb98802c57290394828af60ed95"
Sep 14 03:10:13.910544 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.910518    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c76e1ff08fccb7696d064c303eba92f1ca68dfb98802c57290394828af60ed95"} err="failed to get container status \"c76e1ff08fccb7696d064c303eba92f1ca68dfb98802c57290394828af60ed95\": rpc error: code = NotFound desc = could not find container \"c76e1ff08fccb7696d064c303eba92f1ca68dfb98802c57290394828af60ed95\": container with ID starting with c76e1ff08fccb7696d064c303eba92f1ca68dfb98802c57290394828af60ed95 not found: ID does not exist"
Sep 14 03:10:13.917236 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.917217    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"]
Sep 14 03:10:13.921538 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:13.921520    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-8364a0-openshift-default-64568bf7d8-4scqg"]
Sep 14 03:10:15.466938 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.466908    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"]
Sep 14 03:10:15.467427 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.467161    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj" podUID="c95bb3c7-55b9-4241-a4e7-a34af6b4c53c" containerName="maas-api" containerID="cri-o://5ba4943975b6ec8d8fc4d759e30a6b03f478d6e17fcbd3873524144f49e2e213" gracePeriod=30
Sep 14 03:10:15.711796 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.711773    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:10:15.750312 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.750282    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" path="/var/lib/kubelet/pods/d0478c9a-11f7-4bec-a23d-3a439f3c5d2a/volumes"
Sep 14 03:10:15.825133 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.825105    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-metrics-certs\") pod \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") "
Sep 14 03:10:15.825318 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.825166    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fftmt\" (UniqueName: \"kubernetes.io/projected/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-kube-api-access-fftmt\") pod \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") "
Sep 14 03:10:15.825318 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.825242    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-maas-api-tls\") pod \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") "
Sep 14 03:10:15.825318 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.825277    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-openshift-service-ca\") pod \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\" (UID: \"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c\") "
Sep 14 03:10:15.825718 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.825691    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c95bb3c7-55b9-4241-a4e7-a34af6b4c53c" (UID: "c95bb3c7-55b9-4241-a4e7-a34af6b4c53c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:10:15.827881 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.827852    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c95bb3c7-55b9-4241-a4e7-a34af6b4c53c" (UID: "c95bb3c7-55b9-4241-a4e7-a34af6b4c53c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:10:15.828008 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.827969    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c95bb3c7-55b9-4241-a4e7-a34af6b4c53c" (UID: "c95bb3c7-55b9-4241-a4e7-a34af6b4c53c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:10:15.828082 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.828004    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-kube-api-access-fftmt" (OuterVolumeSpecName: "kube-api-access-fftmt") pod "c95bb3c7-55b9-4241-a4e7-a34af6b4c53c" (UID: "c95bb3c7-55b9-4241-a4e7-a34af6b4c53c"). InnerVolumeSpecName "kube-api-access-fftmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:10:15.907549 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.907513    2579 generic.go:356] "Generic (PLEG): container finished" podID="c95bb3c7-55b9-4241-a4e7-a34af6b4c53c" containerID="5ba4943975b6ec8d8fc4d759e30a6b03f478d6e17fcbd3873524144f49e2e213" exitCode=0
Sep 14 03:10:15.907750 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.907561    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj" event={"ID":"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c","Type":"ContainerDied","Data":"5ba4943975b6ec8d8fc4d759e30a6b03f478d6e17fcbd3873524144f49e2e213"}
Sep 14 03:10:15.907750 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.907577    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"
Sep 14 03:10:15.907750 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.907601    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj" event={"ID":"c95bb3c7-55b9-4241-a4e7-a34af6b4c53c","Type":"ContainerDied","Data":"694ddcd06043946cc89c00a1ee1fd99bdec23dbca68206b36578c73ab93a4944"}
Sep 14 03:10:15.907750 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.907621    2579 scope.go:117] "RemoveContainer" containerID="5ba4943975b6ec8d8fc4d759e30a6b03f478d6e17fcbd3873524144f49e2e213"
Sep 14 03:10:15.918059 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.918042    2579 scope.go:117] "RemoveContainer" containerID="5ba4943975b6ec8d8fc4d759e30a6b03f478d6e17fcbd3873524144f49e2e213"
Sep 14 03:10:15.918351 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:10:15.918332    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ba4943975b6ec8d8fc4d759e30a6b03f478d6e17fcbd3873524144f49e2e213\": container with ID starting with 5ba4943975b6ec8d8fc4d759e30a6b03f478d6e17fcbd3873524144f49e2e213 not found: ID does not exist" containerID="5ba4943975b6ec8d8fc4d759e30a6b03f478d6e17fcbd3873524144f49e2e213"
Sep 14 03:10:15.918403 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.918359    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ba4943975b6ec8d8fc4d759e30a6b03f478d6e17fcbd3873524144f49e2e213"} err="failed to get container status \"5ba4943975b6ec8d8fc4d759e30a6b03f478d6e17fcbd3873524144f49e2e213\": rpc error: code = NotFound desc = could not find container \"5ba4943975b6ec8d8fc4d759e30a6b03f478d6e17fcbd3873524144f49e2e213\": container with ID starting with 5ba4943975b6ec8d8fc4d759e30a6b03f478d6e17fcbd3873524144f49e2e213 not found: ID does not exist"
Sep 14 03:10:15.926521 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.926499    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-metrics-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:15.926521 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.926522    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fftmt\" (UniqueName: \"kubernetes.io/projected/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-kube-api-access-fftmt\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:15.926663 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.926535    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:15.926663 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.926548    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:15.933860 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.933835    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"]
Sep 14 03:10:15.938139 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:15.938116    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-446c5fbd-747bc59999-45lcj"]
Sep 14 03:10:17.744478 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:17.744436    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c95bb3c7-55b9-4241-a4e7-a34af6b4c53c" path="/var/lib/kubelet/pods/c95bb3c7-55b9-4241-a4e7-a34af6b4c53c/volumes"
Sep 14 03:10:22.410485 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:22.410443    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"]
Sep 14 03:10:22.410987 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:22.410644    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h" podUID="43c18a7d-9f1c-4bf3-b813-a13098812df7" containerName="istio-proxy" containerID="cri-o://92b031560ecbe06d2c1caa817f3842d94368b49ba4e41393539d03f590e55521" gracePeriod=30
Sep 14 03:10:25.877208 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:25.877153    2579 patch_prober.go:28] interesting pod/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 03:10:25.877610 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:25.877237    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h" podUID="43c18a7d-9f1c-4bf3-b813-a13098812df7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 03:10:27.670355 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.670332    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:10:27.842751 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.842726    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-workload-socket\") pod \"43c18a7d-9f1c-4bf3-b813-a13098812df7\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") "
Sep 14 03:10:27.842926 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.842763    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-workload-certs\") pod \"43c18a7d-9f1c-4bf3-b813-a13098812df7\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") "
Sep 14 03:10:27.842926 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.842808    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-data\") pod \"43c18a7d-9f1c-4bf3-b813-a13098812df7\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") "
Sep 14 03:10:27.842926 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.842833    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-envoy\") pod \"43c18a7d-9f1c-4bf3-b813-a13098812df7\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") "
Sep 14 03:10:27.842926 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.842860    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-podinfo\") pod \"43c18a7d-9f1c-4bf3-b813-a13098812df7\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") "
Sep 14 03:10:27.842926 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.842895    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-token\") pod \"43c18a7d-9f1c-4bf3-b813-a13098812df7\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") "
Sep 14 03:10:27.843164 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.842951    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzfbr\" (UniqueName: \"kubernetes.io/projected/43c18a7d-9f1c-4bf3-b813-a13098812df7-kube-api-access-vzfbr\") pod \"43c18a7d-9f1c-4bf3-b813-a13098812df7\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") "
Sep 14 03:10:27.843164 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.843025    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "43c18a7d-9f1c-4bf3-b813-a13098812df7" (UID: "43c18a7d-9f1c-4bf3-b813-a13098812df7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:10:27.843164 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.843077    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-data" (OuterVolumeSpecName: "istio-data") pod "43c18a7d-9f1c-4bf3-b813-a13098812df7" (UID: "43c18a7d-9f1c-4bf3-b813-a13098812df7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:10:27.843164 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.843118    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-credential-socket\") pod \"43c18a7d-9f1c-4bf3-b813-a13098812df7\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") "
Sep 14 03:10:27.844098 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.843161    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "43c18a7d-9f1c-4bf3-b813-a13098812df7" (UID: "43c18a7d-9f1c-4bf3-b813-a13098812df7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:10:27.844098 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.843246    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43c18a7d-9f1c-4bf3-b813-a13098812df7-istiod-ca-cert\") pod \"43c18a7d-9f1c-4bf3-b813-a13098812df7\" (UID: \"43c18a7d-9f1c-4bf3-b813-a13098812df7\") "
Sep 14 03:10:27.844098 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.843349    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "43c18a7d-9f1c-4bf3-b813-a13098812df7" (UID: "43c18a7d-9f1c-4bf3-b813-a13098812df7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:10:27.844098 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.843582    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43c18a7d-9f1c-4bf3-b813-a13098812df7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "43c18a7d-9f1c-4bf3-b813-a13098812df7" (UID: "43c18a7d-9f1c-4bf3-b813-a13098812df7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:10:27.844356 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.844108    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:27.844356 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.844131    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43c18a7d-9f1c-4bf3-b813-a13098812df7-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:27.844356 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.844145    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:27.844356 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.844160    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:27.844356 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.844195    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:27.845322 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.845293    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "43c18a7d-9f1c-4bf3-b813-a13098812df7" (UID: "43c18a7d-9f1c-4bf3-b813-a13098812df7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:10:27.845495 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.845464    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-token" (OuterVolumeSpecName: "istio-token") pod "43c18a7d-9f1c-4bf3-b813-a13098812df7" (UID: "43c18a7d-9f1c-4bf3-b813-a13098812df7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:10:27.845608 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.845512    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43c18a7d-9f1c-4bf3-b813-a13098812df7-kube-api-access-vzfbr" (OuterVolumeSpecName: "kube-api-access-vzfbr") pod "43c18a7d-9f1c-4bf3-b813-a13098812df7" (UID: "43c18a7d-9f1c-4bf3-b813-a13098812df7"). InnerVolumeSpecName "kube-api-access-vzfbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:10:27.845608 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.845513    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "43c18a7d-9f1c-4bf3-b813-a13098812df7" (UID: "43c18a7d-9f1c-4bf3-b813-a13098812df7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:10:27.945252 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.945225    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:27.945252 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.945249    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:27.945252 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.945258    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43c18a7d-9f1c-4bf3-b813-a13098812df7-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:27.945484 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.945267    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzfbr\" (UniqueName: \"kubernetes.io/projected/43c18a7d-9f1c-4bf3-b813-a13098812df7-kube-api-access-vzfbr\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:10:27.970446 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.970413    2579 generic.go:356] "Generic (PLEG): container finished" podID="43c18a7d-9f1c-4bf3-b813-a13098812df7" containerID="92b031560ecbe06d2c1caa817f3842d94368b49ba4e41393539d03f590e55521" exitCode=0
Sep 14 03:10:27.970555 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.970463    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"
Sep 14 03:10:27.970555 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.970505    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h" event={"ID":"43c18a7d-9f1c-4bf3-b813-a13098812df7","Type":"ContainerDied","Data":"92b031560ecbe06d2c1caa817f3842d94368b49ba4e41393539d03f590e55521"}
Sep 14 03:10:27.970641 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.970563    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h" event={"ID":"43c18a7d-9f1c-4bf3-b813-a13098812df7","Type":"ContainerDied","Data":"7d8cafbcde5482c33423ef0e5e5a8f2bdd2ab8b0462b3dbf136aeadd420a0d41"}
Sep 14 03:10:27.970641 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.970592    2579 scope.go:117] "RemoveContainer" containerID="92b031560ecbe06d2c1caa817f3842d94368b49ba4e41393539d03f590e55521"
Sep 14 03:10:27.980235 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.980215    2579 scope.go:117] "RemoveContainer" containerID="df5ebe4e36030c499792b1bd0091c757e59b34d2b89cf7392dd554ae43e74683"
Sep 14 03:10:27.988521 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.988504    2579 scope.go:117] "RemoveContainer" containerID="92b031560ecbe06d2c1caa817f3842d94368b49ba4e41393539d03f590e55521"
Sep 14 03:10:27.988766 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:10:27.988748    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92b031560ecbe06d2c1caa817f3842d94368b49ba4e41393539d03f590e55521\": container with ID starting with 92b031560ecbe06d2c1caa817f3842d94368b49ba4e41393539d03f590e55521 not found: ID does not exist" containerID="92b031560ecbe06d2c1caa817f3842d94368b49ba4e41393539d03f590e55521"
Sep 14 03:10:27.988820 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.988774    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92b031560ecbe06d2c1caa817f3842d94368b49ba4e41393539d03f590e55521"} err="failed to get container status \"92b031560ecbe06d2c1caa817f3842d94368b49ba4e41393539d03f590e55521\": rpc error: code = NotFound desc = could not find container \"92b031560ecbe06d2c1caa817f3842d94368b49ba4e41393539d03f590e55521\": container with ID starting with 92b031560ecbe06d2c1caa817f3842d94368b49ba4e41393539d03f590e55521 not found: ID does not exist"
Sep 14 03:10:27.988820 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.988791    2579 scope.go:117] "RemoveContainer" containerID="df5ebe4e36030c499792b1bd0091c757e59b34d2b89cf7392dd554ae43e74683"
Sep 14 03:10:27.989037 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:10:27.989020    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df5ebe4e36030c499792b1bd0091c757e59b34d2b89cf7392dd554ae43e74683\": container with ID starting with df5ebe4e36030c499792b1bd0091c757e59b34d2b89cf7392dd554ae43e74683 not found: ID does not exist" containerID="df5ebe4e36030c499792b1bd0091c757e59b34d2b89cf7392dd554ae43e74683"
Sep 14 03:10:27.989090 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.989045    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df5ebe4e36030c499792b1bd0091c757e59b34d2b89cf7392dd554ae43e74683"} err="failed to get container status \"df5ebe4e36030c499792b1bd0091c757e59b34d2b89cf7392dd554ae43e74683\": rpc error: code = NotFound desc = could not find container \"df5ebe4e36030c499792b1bd0091c757e59b34d2b89cf7392dd554ae43e74683\": container with ID starting with df5ebe4e36030c499792b1bd0091c757e59b34d2b89cf7392dd554ae43e74683 not found: ID does not exist"
Sep 14 03:10:27.993548 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.993527    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"]
Sep 14 03:10:27.998031 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:27.998009    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-446c5fbd-openshift-default-648f5bc69b-x5s2h"]
Sep 14 03:10:29.743787 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:29.743758    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43c18a7d-9f1c-4bf3-b813-a13098812df7" path="/var/lib/kubelet/pods/43c18a7d-9f1c-4bf3-b813-a13098812df7/volumes"
Sep 14 03:10:36.417040 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.417003    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"]
Sep 14 03:10:36.417642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.417404    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" containerName="istio-proxy"
Sep 14 03:10:36.417642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.417419    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" containerName="istio-proxy"
Sep 14 03:10:36.417642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.417428    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43c18a7d-9f1c-4bf3-b813-a13098812df7" containerName="istio-proxy"
Sep 14 03:10:36.417642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.417434    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="43c18a7d-9f1c-4bf3-b813-a13098812df7" containerName="istio-proxy"
Sep 14 03:10:36.417642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.417446    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43c18a7d-9f1c-4bf3-b813-a13098812df7" containerName="istio-proxy"
Sep 14 03:10:36.417642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.417451    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="43c18a7d-9f1c-4bf3-b813-a13098812df7" containerName="istio-proxy"
Sep 14 03:10:36.417642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.417465    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c95bb3c7-55b9-4241-a4e7-a34af6b4c53c" containerName="maas-api"
Sep 14 03:10:36.417642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.417470    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c95bb3c7-55b9-4241-a4e7-a34af6b4c53c" containerName="maas-api"
Sep 14 03:10:36.417642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.417479    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" containerName="istio-proxy"
Sep 14 03:10:36.417642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.417484    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" containerName="istio-proxy"
Sep 14 03:10:36.417642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.417544    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" containerName="istio-proxy"
Sep 14 03:10:36.417642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.417552    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="43c18a7d-9f1c-4bf3-b813-a13098812df7" containerName="istio-proxy"
Sep 14 03:10:36.417642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.417557    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c95bb3c7-55b9-4241-a4e7-a34af6b4c53c" containerName="maas-api"
Sep 14 03:10:36.420950 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.420926    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.425342 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.425318    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-ce5b9b-openshift-default-dockercfg-k48cv\""
Sep 14 03:10:36.441185 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.441149    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"]
Sep 14 03:10:36.523532 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.523498    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-envoy\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.523691 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.523539    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-credential-socket\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.523691 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.523564    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-podinfo\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.523691 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.523582    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dw87\" (UniqueName: \"kubernetes.io/projected/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-kube-api-access-7dw87\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.523691 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.523674    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-workload-certs\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.523872 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.523716    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-token\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.523872 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.523756    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istiod-ca-cert\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.523872 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.523776    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-workload-socket\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.523872 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.523793    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-data\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.625262 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.625231    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-workload-certs\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.625436 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.625278    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-token\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.625436 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.625404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istiod-ca-cert\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.625557 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.625445    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-workload-socket\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.625557 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.625471    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-data\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.625557 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.625541    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-envoy\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.625713 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.625585    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-credential-socket\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.625713 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.625626    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-podinfo\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.625713 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.625655    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dw87\" (UniqueName: \"kubernetes.io/projected/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-kube-api-access-7dw87\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.625713 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.625672    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-workload-certs\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.625920 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.625805    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-workload-socket\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.625967 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.625912    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-data\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.626100 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.626080    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istiod-ca-cert\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.626161 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.626143    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-credential-socket\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.627790 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.627771    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-envoy\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.628090 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.628073    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-podinfo\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.634012 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.633983    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dw87\" (UniqueName: \"kubernetes.io/projected/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-kube-api-access-7dw87\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.634341 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.634323    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-token\") pod \"e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") " pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:36.732148 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:36.732092    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:37.034394 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:37.034368    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"]
Sep 14 03:10:37.035708 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:10:37.035684    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd8e1b0e_92cf_4e3c_9952_d9248fc041eb.slice/crio-04fc05068df8559cc912bca343c408a379395ad6c93857cb66a523fa7da8537a WatchSource:0}: Error finding container 04fc05068df8559cc912bca343c408a379395ad6c93857cb66a523fa7da8537a: Status 404 returned error can't find the container with id 04fc05068df8559cc912bca343c408a379395ad6c93857cb66a523fa7da8537a
Sep 14 03:10:37.037737 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:37.037702    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:10:37.037832 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:37.037764    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:10:37.037832 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:37.037791    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:10:38.027130 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:38.027097    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9" event={"ID":"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb","Type":"ContainerStarted","Data":"15d648cb05738e9010c7f72cbaea1ce3528ea3651c228ea53dcf2f37f81ba47b"}
Sep 14 03:10:38.027530 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:38.027151    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9" event={"ID":"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb","Type":"ContainerStarted","Data":"04fc05068df8559cc912bca343c408a379395ad6c93857cb66a523fa7da8537a"}
Sep 14 03:10:38.051266 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:38.051217    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9" podStartSLOduration=2.051200658 podStartE2EDuration="2.051200658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:10:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:10:38.051075889 +0000 UTC m=+1100.837137805" watchObservedRunningTime="2026-09-14 03:10:38.051200658 +0000 UTC m=+1100.837262564"
Sep 14 03:10:38.733204 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:38.733135    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:38.738850 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:38.738824    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:39.033757 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:39.033729    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:39.034712 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:39.034695    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:10:43.847385 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:43.847352    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"]
Sep 14 03:10:43.847865 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:43.847851    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0478c9a-11f7-4bec-a23d-3a439f3c5d2a" containerName="istio-proxy"
Sep 14 03:10:43.847918 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:43.847868    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="43c18a7d-9f1c-4bf3-b813-a13098812df7" containerName="istio-proxy"
Sep 14 03:10:43.850494 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:43.850478    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"
Sep 14 03:10:43.853491 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:43.853469    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-ce5b9b\""
Sep 14 03:10:43.853788 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:43.853774    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-ce5b9b-dockercfg-mwrcw\""
Sep 14 03:10:43.861265 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:43.861242    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"]
Sep 14 03:10:43.892460 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:43.892433    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c55cf6b4-ff06-406e-b64b-548320256283-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br\" (UID: \"c55cf6b4-ff06-406e-b64b-548320256283\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"
Sep 14 03:10:43.892581 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:43.892553    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkpbx\" (UniqueName: \"kubernetes.io/projected/c55cf6b4-ff06-406e-b64b-548320256283-kube-api-access-gkpbx\") pod \"payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br\" (UID: \"c55cf6b4-ff06-406e-b64b-548320256283\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"
Sep 14 03:10:43.993639 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:43.993603    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkpbx\" (UniqueName: \"kubernetes.io/projected/c55cf6b4-ff06-406e-b64b-548320256283-kube-api-access-gkpbx\") pod \"payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br\" (UID: \"c55cf6b4-ff06-406e-b64b-548320256283\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"
Sep 14 03:10:43.993826 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:43.993727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c55cf6b4-ff06-406e-b64b-548320256283-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br\" (UID: \"c55cf6b4-ff06-406e-b64b-548320256283\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"
Sep 14 03:10:43.994641 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:43.994616    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c55cf6b4-ff06-406e-b64b-548320256283-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br\" (UID: \"c55cf6b4-ff06-406e-b64b-548320256283\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"
Sep 14 03:10:44.002346 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:44.002323    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkpbx\" (UniqueName: \"kubernetes.io/projected/c55cf6b4-ff06-406e-b64b-548320256283-kube-api-access-gkpbx\") pod \"payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br\" (UID: \"c55cf6b4-ff06-406e-b64b-548320256283\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"
Sep 14 03:10:44.162462 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:44.162374    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"
Sep 14 03:10:44.438753 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:44.438731    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"]
Sep 14 03:10:44.440743 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:10:44.440716    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc55cf6b4_ff06_406e_b64b_548320256283.slice/crio-905fe7a72f05fce0b1645e10c2c46d2d8d0deee0c562bb165f8d7d774ec21468 WatchSource:0}: Error finding container 905fe7a72f05fce0b1645e10c2c46d2d8d0deee0c562bb165f8d7d774ec21468: Status 404 returned error can't find the container with id 905fe7a72f05fce0b1645e10c2c46d2d8d0deee0c562bb165f8d7d774ec21468
Sep 14 03:10:45.066465 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:45.066427    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br" event={"ID":"c55cf6b4-ff06-406e-b64b-548320256283","Type":"ContainerStarted","Data":"98ff40ec7de6cbcb80aa4709c8608edbb99069e8d37ff3684078d930b980078c"}
Sep 14 03:10:45.066841 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:45.066486    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br" event={"ID":"c55cf6b4-ff06-406e-b64b-548320256283","Type":"ContainerStarted","Data":"905fe7a72f05fce0b1645e10c2c46d2d8d0deee0c562bb165f8d7d774ec21468"}
Sep 14 03:10:45.066841 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:45.066554    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"
Sep 14 03:10:45.085352 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:45.085309    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br" podStartSLOduration=2.085297806 podStartE2EDuration="2.085297806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:10:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:10:45.084030129 +0000 UTC m=+1107.870092043" watchObservedRunningTime="2026-09-14 03:10:45.085297806 +0000 UTC m=+1107.871359779"
Sep 14 03:10:56.073795 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:10:56.073764    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"
Sep 14 03:11:02.165990 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:02.165955    2579 generic.go:356] "Generic (PLEG): container finished" podID="bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" containerID="15d648cb05738e9010c7f72cbaea1ce3528ea3651c228ea53dcf2f37f81ba47b" exitCode=0
Sep 14 03:11:02.166435 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:02.166028    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9" event={"ID":"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb","Type":"ContainerDied","Data":"15d648cb05738e9010c7f72cbaea1ce3528ea3651c228ea53dcf2f37f81ba47b"}
Sep 14 03:11:02.166480 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:02.166448    2579 scope.go:117] "RemoveContainer" containerID="15d648cb05738e9010c7f72cbaea1ce3528ea3651c228ea53dcf2f37f81ba47b"
Sep 14 03:11:02.167624 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:02.167581    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:11:02.167732 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:02.167665    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:11:02.167732 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:02.167694    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:11:03.172851 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:03.172819    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9" event={"ID":"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb","Type":"ContainerStarted","Data":"c9ad893fe4d4d8cfe7de824e9125d0bb527b666501e661df49c5854004ec9539"}
Sep 14 03:11:03.732340 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:03.732304    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:11:03.732525 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:03.732508    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 14 03:11:03.732578 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:03.732562    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9" podUID="bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 14 03:11:04.732972 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:04.732941    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 14 03:11:04.733404 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:04.733009    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9" podUID="bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 14 03:11:05.732538 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:05.732503    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 14 03:11:05.732720 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:05.732561    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9" podUID="bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 14 03:11:06.733516 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:06.733479    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 14 03:11:06.733907 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:06.733548    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9" podUID="bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 14 03:11:07.732656 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:07.732619    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 14 03:11:07.732902 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:07.732677    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9" podUID="bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 14 03:11:08.737924 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:08.737895    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:11:08.738306 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:08.738266    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:11:08.738888 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:08.738870    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:11:10.652493 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.652464    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"]
Sep 14 03:11:10.655312 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.655292    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.658313 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.658292    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-17d8b8-openshift-default-dockercfg-dmc9q\""
Sep 14 03:11:10.681977 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.681954    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"]
Sep 14 03:11:10.755765 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.755726    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b13dad7c-b629-4395-b830-005b15777742-istiod-ca-cert\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.756110 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.756079    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-istio-envoy\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.756405 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.756369    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b13dad7c-b629-4395-b830-005b15777742-istio-token\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.756598 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.756573    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smbnc\" (UniqueName: \"kubernetes.io/projected/b13dad7c-b629-4395-b830-005b15777742-kube-api-access-smbnc\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.756749 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.756735    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-credential-socket\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.756863 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.756848    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-workload-socket\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.756981 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.756966    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-istio-data\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.757099 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.757085    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b13dad7c-b629-4395-b830-005b15777742-istio-podinfo\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.757278 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.757243    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-workload-certs\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.858306 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.858274    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b13dad7c-b629-4395-b830-005b15777742-istio-token\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.858490 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.858328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smbnc\" (UniqueName: \"kubernetes.io/projected/b13dad7c-b629-4395-b830-005b15777742-kube-api-access-smbnc\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.858490 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.858365    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-credential-socket\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.858490 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.858382    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-workload-socket\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.858490 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.858404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-istio-data\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.858490 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.858430    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b13dad7c-b629-4395-b830-005b15777742-istio-podinfo\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.858772 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.858580    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-workload-certs\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.858772 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.858648    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b13dad7c-b629-4395-b830-005b15777742-istiod-ca-cert\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.858772 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.858710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-istio-envoy\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.858918 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.858804    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-workload-socket\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.858918 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.858855    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-istio-data\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.859124 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.859104    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-workload-certs\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.859217 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.859129    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-credential-socket\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.859465 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.859440    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b13dad7c-b629-4395-b830-005b15777742-istiod-ca-cert\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.860984 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.860959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-istio-envoy\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.861087 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.861060    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b13dad7c-b629-4395-b830-005b15777742-istio-podinfo\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.868988 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.868964    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smbnc\" (UniqueName: \"kubernetes.io/projected/b13dad7c-b629-4395-b830-005b15777742-kube-api-access-smbnc\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.869398 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.869382    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b13dad7c-b629-4395-b830-005b15777742-istio-token\") pod \"e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") " pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:10.967494 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:10.967416    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:11.256006 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:11.255981    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"]
Sep 14 03:11:11.257713 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:11:11.257682    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb13dad7c_b629_4395_b830_005b15777742.slice/crio-45bfaf90d6f6ded9d880d934394331ced9a4ee9bb28fd9996a5b2424be07dfd2 WatchSource:0}: Error finding container 45bfaf90d6f6ded9d880d934394331ced9a4ee9bb28fd9996a5b2424be07dfd2: Status 404 returned error can't find the container with id 45bfaf90d6f6ded9d880d934394331ced9a4ee9bb28fd9996a5b2424be07dfd2
Sep 14 03:11:11.259880 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:11.259853    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:11:11.259944 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:11.259920    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:11:11.259981 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:11.259950    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:11:12.226631 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:12.226588    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk" event={"ID":"b13dad7c-b629-4395-b830-005b15777742","Type":"ContainerStarted","Data":"f161e3951af4f8e85765737e2c9be145cda43a36bfb5e9a97e9ee496f4f8545d"}
Sep 14 03:11:12.227013 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:12.226687    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk" event={"ID":"b13dad7c-b629-4395-b830-005b15777742","Type":"ContainerStarted","Data":"45bfaf90d6f6ded9d880d934394331ced9a4ee9bb28fd9996a5b2424be07dfd2"}
Sep 14 03:11:12.235395 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:12.235370    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl"]
Sep 14 03:11:12.238039 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:12.238022    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl"
Sep 14 03:11:12.240587 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:12.240566    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:11:12.240697 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:12.240568    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:11:12.248119 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:12.248092    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl"]
Sep 14 03:11:12.250754 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:12.250708    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk" podStartSLOduration=2.25069137 podStartE2EDuration="2.25069137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:11:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:11:12.249934913 +0000 UTC m=+1135.035996825" watchObservedRunningTime="2026-09-14 03:11:12.25069137 +0000 UTC m=+1135.036753284"
Sep 14 03:11:12.373229 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:12.373190    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77806e2e-a920-4fe3-857e-bd86e1315b9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl\" (UID: \"77806e2e-a920-4fe3-857e-bd86e1315b9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl"
Sep 14 03:11:12.475031 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:12.474994    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77806e2e-a920-4fe3-857e-bd86e1315b9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl\" (UID: \"77806e2e-a920-4fe3-857e-bd86e1315b9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl"
Sep 14 03:11:12.475610 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:12.475587    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77806e2e-a920-4fe3-857e-bd86e1315b9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl\" (UID: \"77806e2e-a920-4fe3-857e-bd86e1315b9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl"
Sep 14 03:11:12.549646 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:12.549616    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl"
Sep 14 03:11:12.831023 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:12.831001    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl"]
Sep 14 03:11:12.832732 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:11:12.832698    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77806e2e_a920_4fe3_857e_bd86e1315b9b.slice/crio-fd766beb26be3874d66f53174d0526fc02c6a984f52368c0668f686c10fa841b WatchSource:0}: Error finding container fd766beb26be3874d66f53174d0526fc02c6a984f52368c0668f686c10fa841b: Status 404 returned error can't find the container with id fd766beb26be3874d66f53174d0526fc02c6a984f52368c0668f686c10fa841b
Sep 14 03:11:12.968305 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:12.968278    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:12.973382 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:12.973362    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:13.232933 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:13.232853    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl" event={"ID":"77806e2e-a920-4fe3-857e-bd86e1315b9b","Type":"ContainerStarted","Data":"c35b338c563948abe87736abd63795c205b9116a435f9f36c07f8885fb4f73f6"}
Sep 14 03:11:13.232933 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:13.232900    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl" event={"ID":"77806e2e-a920-4fe3-857e-bd86e1315b9b","Type":"ContainerStarted","Data":"fd766beb26be3874d66f53174d0526fc02c6a984f52368c0668f686c10fa841b"}
Sep 14 03:11:13.233359 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:13.233245    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:13.233986 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:13.233972    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:13.252022 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:13.251982    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl" podStartSLOduration=1.251968754 podStartE2EDuration="1.251968754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:11:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:11:13.249801828 +0000 UTC m=+1136.035863754" watchObservedRunningTime="2026-09-14 03:11:13.251968754 +0000 UTC m=+1136.038030718"
Sep 14 03:11:14.239642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:14.239610    2579 generic.go:356] "Generic (PLEG): container finished" podID="77806e2e-a920-4fe3-857e-bd86e1315b9b" containerID="c35b338c563948abe87736abd63795c205b9116a435f9f36c07f8885fb4f73f6" exitCode=0
Sep 14 03:11:14.240024 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:14.239692    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl" event={"ID":"77806e2e-a920-4fe3-857e-bd86e1315b9b","Type":"ContainerDied","Data":"c35b338c563948abe87736abd63795c205b9116a435f9f36c07f8885fb4f73f6"}
Sep 14 03:11:15.385789 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:15.385767    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl"
Sep 14 03:11:15.405728 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:15.405671    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77806e2e-a920-4fe3-857e-bd86e1315b9b-maas-api-service-ca\") pod \"77806e2e-a920-4fe3-857e-bd86e1315b9b\" (UID: \"77806e2e-a920-4fe3-857e-bd86e1315b9b\") "
Sep 14 03:11:15.406273 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:15.406249    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77806e2e-a920-4fe3-857e-bd86e1315b9b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "77806e2e-a920-4fe3-857e-bd86e1315b9b" (UID: "77806e2e-a920-4fe3-857e-bd86e1315b9b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:11:15.506625 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:15.506545    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77806e2e-a920-4fe3-857e-bd86e1315b9b-maas-api-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:11:16.252054 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:16.252025    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl"
Sep 14 03:11:16.252223 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:16.252048    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl" event={"ID":"77806e2e-a920-4fe3-857e-bd86e1315b9b","Type":"ContainerDied","Data":"fd766beb26be3874d66f53174d0526fc02c6a984f52368c0668f686c10fa841b"}
Sep 14 03:11:16.252223 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:16.252094    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd766beb26be3874d66f53174d0526fc02c6a984f52368c0668f686c10fa841b"
Sep 14 03:11:18.898491 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:18.898447    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"]
Sep 14 03:11:18.898976 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:18.898844    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77806e2e-a920-4fe3-857e-bd86e1315b9b" containerName="revoke-keys"
Sep 14 03:11:18.898976 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:18.898855    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="77806e2e-a920-4fe3-857e-bd86e1315b9b" containerName="revoke-keys"
Sep 14 03:11:18.898976 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:18.898931    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="77806e2e-a920-4fe3-857e-bd86e1315b9b" containerName="revoke-keys"
Sep 14 03:11:18.901852 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:18.901832    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"
Sep 14 03:11:18.904998 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:18.904982    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-17d8b8\""
Sep 14 03:11:18.905099 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:18.905039    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-17d8b8-dockercfg-z64v9\""
Sep 14 03:11:18.912340 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:18.912322    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"]
Sep 14 03:11:18.931427 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:18.931392    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djvcn\" (UniqueName: \"kubernetes.io/projected/54182426-00ba-4567-9347-054e89dd65dc-kube-api-access-djvcn\") pod \"payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz\" (UID: \"54182426-00ba-4567-9347-054e89dd65dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"
Sep 14 03:11:18.931539 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:18.931516    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54182426-00ba-4567-9347-054e89dd65dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz\" (UID: \"54182426-00ba-4567-9347-054e89dd65dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"
Sep 14 03:11:19.032403 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:19.032367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djvcn\" (UniqueName: \"kubernetes.io/projected/54182426-00ba-4567-9347-054e89dd65dc-kube-api-access-djvcn\") pod \"payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz\" (UID: \"54182426-00ba-4567-9347-054e89dd65dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"
Sep 14 03:11:19.032595 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:19.032463    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54182426-00ba-4567-9347-054e89dd65dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz\" (UID: \"54182426-00ba-4567-9347-054e89dd65dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"
Sep 14 03:11:19.033168 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:19.033147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54182426-00ba-4567-9347-054e89dd65dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz\" (UID: \"54182426-00ba-4567-9347-054e89dd65dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"
Sep 14 03:11:19.041287 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:19.041265    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djvcn\" (UniqueName: \"kubernetes.io/projected/54182426-00ba-4567-9347-054e89dd65dc-kube-api-access-djvcn\") pod \"payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz\" (UID: \"54182426-00ba-4567-9347-054e89dd65dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"
Sep 14 03:11:19.213324 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:19.213230    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"
Sep 14 03:11:19.499570 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:19.499543    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"]
Sep 14 03:11:19.501446 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:11:19.501419    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54182426_00ba_4567_9347_054e89dd65dc.slice/crio-efb1f56a51fb5db8b9d52d4f3f35fdc8dd4fb18bd136ee7e1cb6b6a941f3a515 WatchSource:0}: Error finding container efb1f56a51fb5db8b9d52d4f3f35fdc8dd4fb18bd136ee7e1cb6b6a941f3a515: Status 404 returned error can't find the container with id efb1f56a51fb5db8b9d52d4f3f35fdc8dd4fb18bd136ee7e1cb6b6a941f3a515
Sep 14 03:11:20.275687 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:20.275654    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz" event={"ID":"54182426-00ba-4567-9347-054e89dd65dc","Type":"ContainerStarted","Data":"333f227f32b83e044752fb019bcadb970b0527b9116876b3a72184b4a5df7783"}
Sep 14 03:11:20.276092 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:20.275704    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz" event={"ID":"54182426-00ba-4567-9347-054e89dd65dc","Type":"ContainerStarted","Data":"efb1f56a51fb5db8b9d52d4f3f35fdc8dd4fb18bd136ee7e1cb6b6a941f3a515"}
Sep 14 03:11:20.276092 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:20.275755    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"
Sep 14 03:11:20.294637 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:20.294584    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz" podStartSLOduration=2.294563871 podStartE2EDuration="2.294563871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:11:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:11:20.293671251 +0000 UTC m=+1143.079733166" watchObservedRunningTime="2026-09-14 03:11:20.294563871 +0000 UTC m=+1143.080625786"
Sep 14 03:11:23.113874 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:23.113840    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"]
Sep 14 03:11:23.114353 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:23.114085    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt" podUID="6712cbe0-242b-4630-b044-6daa82d59a6a" containerName="payload-processing" containerID="cri-o://27f1db36f1b396ff062ec3c9dcc9f32e7d148f3dae1b81cb3a32bb53b88d0dc6" gracePeriod=30
Sep 14 03:11:23.294007 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:23.293875    2579 generic.go:356] "Generic (PLEG): container finished" podID="6712cbe0-242b-4630-b044-6daa82d59a6a" containerID="27f1db36f1b396ff062ec3c9dcc9f32e7d148f3dae1b81cb3a32bb53b88d0dc6" exitCode=0
Sep 14 03:11:23.294007 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:23.293949    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt" event={"ID":"6712cbe0-242b-4630-b044-6daa82d59a6a","Type":"ContainerDied","Data":"27f1db36f1b396ff062ec3c9dcc9f32e7d148f3dae1b81cb3a32bb53b88d0dc6"}
Sep 14 03:11:23.309296 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:23.309253    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl"]
Sep 14 03:11:23.315060 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:23.315032    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f002a1-cq6bl"]
Sep 14 03:11:23.383832 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:23.383809    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"
Sep 14 03:11:23.474198 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:23.473658    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9x8gb\" (UniqueName: \"kubernetes.io/projected/6712cbe0-242b-4630-b044-6daa82d59a6a-kube-api-access-9x8gb\") pod \"6712cbe0-242b-4630-b044-6daa82d59a6a\" (UID: \"6712cbe0-242b-4630-b044-6daa82d59a6a\") "
Sep 14 03:11:23.474198 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:23.473746    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6712cbe0-242b-4630-b044-6daa82d59a6a-plugins-config-volume\") pod \"6712cbe0-242b-4630-b044-6daa82d59a6a\" (UID: \"6712cbe0-242b-4630-b044-6daa82d59a6a\") "
Sep 14 03:11:23.474472 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:23.474417    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6712cbe0-242b-4630-b044-6daa82d59a6a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6712cbe0-242b-4630-b044-6daa82d59a6a" (UID: "6712cbe0-242b-4630-b044-6daa82d59a6a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:11:23.478568 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:23.478025    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6712cbe0-242b-4630-b044-6daa82d59a6a-kube-api-access-9x8gb" (OuterVolumeSpecName: "kube-api-access-9x8gb") pod "6712cbe0-242b-4630-b044-6daa82d59a6a" (UID: "6712cbe0-242b-4630-b044-6daa82d59a6a"). InnerVolumeSpecName "kube-api-access-9x8gb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:11:23.575098 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:23.575066    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6712cbe0-242b-4630-b044-6daa82d59a6a-plugins-config-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:11:23.575098 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:23.575094    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9x8gb\" (UniqueName: \"kubernetes.io/projected/6712cbe0-242b-4630-b044-6daa82d59a6a-kube-api-access-9x8gb\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:11:23.744576 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:23.744548    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77806e2e-a920-4fe3-857e-bd86e1315b9b" path="/var/lib/kubelet/pods/77806e2e-a920-4fe3-857e-bd86e1315b9b/volumes"
Sep 14 03:11:24.302130 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:24.302102    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"
Sep 14 03:11:24.302661 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:24.302129    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt" event={"ID":"6712cbe0-242b-4630-b044-6daa82d59a6a","Type":"ContainerDied","Data":"a1fa41c60baa043e69c60ae031edcee5450ac71b2ada4183c21d60ef5948dd63"}
Sep 14 03:11:24.302661 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:24.302220    2579 scope.go:117] "RemoveContainer" containerID="27f1db36f1b396ff062ec3c9dcc9f32e7d148f3dae1b81cb3a32bb53b88d0dc6"
Sep 14 03:11:24.321379 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:24.321354    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"]
Sep 14 03:11:24.323223 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:24.323204    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-f002a1-8b996b58f-f2rkt"]
Sep 14 03:11:25.749526 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:25.749485    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6712cbe0-242b-4630-b044-6daa82d59a6a" path="/var/lib/kubelet/pods/6712cbe0-242b-4630-b044-6daa82d59a6a/volumes"
Sep 14 03:11:31.283589 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:31.283562    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"
Sep 14 03:11:38.387773 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.387739    2579 generic.go:356] "Generic (PLEG): container finished" podID="b13dad7c-b629-4395-b830-005b15777742" containerID="f161e3951af4f8e85765737e2c9be145cda43a36bfb5e9a97e9ee496f4f8545d" exitCode=0
Sep 14 03:11:38.388253 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.387813    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk" event={"ID":"b13dad7c-b629-4395-b830-005b15777742","Type":"ContainerDied","Data":"f161e3951af4f8e85765737e2c9be145cda43a36bfb5e9a97e9ee496f4f8545d"}
Sep 14 03:11:38.388306 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.388272    2579 scope.go:117] "RemoveContainer" containerID="f161e3951af4f8e85765737e2c9be145cda43a36bfb5e9a97e9ee496f4f8545d"
Sep 14 03:11:38.389406 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.389381    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:11:38.389471 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.389452    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:11:38.389517 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.389491    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:11:38.414668 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.414647    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"]
Sep 14 03:11:38.415085 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.415072    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6712cbe0-242b-4630-b044-6daa82d59a6a" containerName="payload-processing"
Sep 14 03:11:38.415158 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.415088    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6712cbe0-242b-4630-b044-6daa82d59a6a" containerName="payload-processing"
Sep 14 03:11:38.415240 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.415158    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6712cbe0-242b-4630-b044-6daa82d59a6a" containerName="payload-processing"
Sep 14 03:11:38.419212 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.418602    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.422022 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.422002    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-12ecd131-openshift-default-dockercfg-8ckvc\""
Sep 14 03:11:38.436836 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.436810    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"]
Sep 14 03:11:38.517912 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.517868    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-envoy\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.518078 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.518033    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-credential-socket\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.518078 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.518074    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-workload-certs\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.518217 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.518198    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-data\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.518288 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.518236    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/265572f5-7a69-43a0-879d-62abe5b85d5f-istiod-ca-cert\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.518412 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.518393    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-podinfo\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.518477 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.518464    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-workload-socket\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.518531 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.518492    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzmg9\" (UniqueName: \"kubernetes.io/projected/265572f5-7a69-43a0-879d-62abe5b85d5f-kube-api-access-zzmg9\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.518582 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.518564    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-token\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.619848 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.619763    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-workload-certs\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.619848 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.619838    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-data\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.620028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.619864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/265572f5-7a69-43a0-879d-62abe5b85d5f-istiod-ca-cert\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.620028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.619933    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-podinfo\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.620028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.619997    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-workload-socket\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.620195 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.620049    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzmg9\" (UniqueName: \"kubernetes.io/projected/265572f5-7a69-43a0-879d-62abe5b85d5f-kube-api-access-zzmg9\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.620195 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.620112    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-token\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.620195 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.620149    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-envoy\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.620356 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.620240    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-credential-socket\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.620415 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.620387    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-data\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.620491 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.620457    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-workload-certs\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.620702 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.620681    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-credential-socket\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.621008 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.620953    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-workload-socket\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.621279 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.621202    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/265572f5-7a69-43a0-879d-62abe5b85d5f-istiod-ca-cert\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.622756 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.622730    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-podinfo\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.622849 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.622792    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-envoy\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.632799 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.632782    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-token\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.640088 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.640042    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzmg9\" (UniqueName: \"kubernetes.io/projected/265572f5-7a69-43a0-879d-62abe5b85d5f-kube-api-access-zzmg9\") pod \"e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") " pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:38.747687 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:38.747656    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:39.038698 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:39.038657    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"]
Sep 14 03:11:39.040569 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:11:39.040539    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod265572f5_7a69_43a0_879d_62abe5b85d5f.slice/crio-21d7343fb7ebd3b514cd565326bd44e86f515584970aea441cf2fa5b9c1b5de0 WatchSource:0}: Error finding container 21d7343fb7ebd3b514cd565326bd44e86f515584970aea441cf2fa5b9c1b5de0: Status 404 returned error can't find the container with id 21d7343fb7ebd3b514cd565326bd44e86f515584970aea441cf2fa5b9c1b5de0
Sep 14 03:11:39.042924 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:39.042895    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:11:39.042995 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:39.042959    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:11:39.043037 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:39.042998    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:11:39.398750 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:39.398652    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk" event={"ID":"b13dad7c-b629-4395-b830-005b15777742","Type":"ContainerStarted","Data":"86a2339c61db33642434518f6afb89247e8e2b7e83b32fd589ae4f60e41d9138"}
Sep 14 03:11:39.401887 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:39.401828    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w" event={"ID":"265572f5-7a69-43a0-879d-62abe5b85d5f","Type":"ContainerStarted","Data":"eb28d3ef1bd9e7427320045927585b3023de391570cc2853f6f85c4bcd17b39a"}
Sep 14 03:11:39.402043 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:39.401906    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w" event={"ID":"265572f5-7a69-43a0-879d-62abe5b85d5f","Type":"ContainerStarted","Data":"21d7343fb7ebd3b514cd565326bd44e86f515584970aea441cf2fa5b9c1b5de0"}
Sep 14 03:11:39.442328 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:39.442269    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w" podStartSLOduration=1.442248381 podStartE2EDuration="1.442248381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:11:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:11:39.439647594 +0000 UTC m=+1162.225709508" watchObservedRunningTime="2026-09-14 03:11:39.442248381 +0000 UTC m=+1162.228310294"
Sep 14 03:11:39.747831 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:39.747803    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:39.967640 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:39.967605    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:39.969010 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:39.968983    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 14 03:11:39.969136 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:39.969036    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk" podUID="b13dad7c-b629-4395-b830-005b15777742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 14 03:11:40.753888 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:40.753849    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:40.968839 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:40.968807    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 14 03:11:40.969006 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:40.968865    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk" podUID="b13dad7c-b629-4395-b830-005b15777742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 14 03:11:41.418479 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:41.418441    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:41.419293 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:41.419270    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:11:41.968663 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:41.968624    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 14 03:11:41.969061 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:41.968701    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk" podUID="b13dad7c-b629-4395-b830-005b15777742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 14 03:11:42.968624 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:42.968587    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 14 03:11:42.968802 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:42.968647    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk" podUID="b13dad7c-b629-4395-b830-005b15777742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 14 03:11:43.234195 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:43.234082    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:43.968482 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:43.968443    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 14 03:11:43.968698 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:43.968512    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk" podUID="b13dad7c-b629-4395-b830-005b15777742" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 14 03:11:44.973239 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:44.973210    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:44.974299 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:44.974279    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:11:53.571526 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:53.571494    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"]
Sep 14 03:11:53.571948 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:53.571686    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq" podUID="7d022749-c77d-43f1-97fe-21f6a0cbfa2b" containerName="istio-proxy" containerID="cri-o://6ac28a3f5ff46fbe0cf78f9bf72bf6d8655c3586852a827d568e721f878f5529" gracePeriod=30
Sep 14 03:11:55.876774 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:55.876738    2579 patch_prober.go:28] interesting pod/e2e-noten-f16030-openshift-default-699d768b48-gqkcq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 03:11:55.877137 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:55.876827    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq" podUID="7d022749-c77d-43f1-97fe-21f6a0cbfa2b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 03:11:58.824588 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.824560    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:11:58.920960 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.920883    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-podinfo\") pod \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") "
Sep 14 03:11:58.920960 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.920917    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-workload-certs\") pod \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") "
Sep 14 03:11:58.920960 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.920954    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-envoy\") pod \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") "
Sep 14 03:11:58.921243 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.920981    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fn8n4\" (UniqueName: \"kubernetes.io/projected/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-kube-api-access-fn8n4\") pod \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") "
Sep 14 03:11:58.921243 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.921002    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-workload-socket\") pod \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") "
Sep 14 03:11:58.921243 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.921061    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-credential-socket\") pod \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") "
Sep 14 03:11:58.921243 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.921092    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istiod-ca-cert\") pod \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") "
Sep 14 03:11:58.921243 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.921149    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7d022749-c77d-43f1-97fe-21f6a0cbfa2b" (UID: "7d022749-c77d-43f1-97fe-21f6a0cbfa2b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:11:58.921534 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.921252    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-token\") pod \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") "
Sep 14 03:11:58.921534 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.921281    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-data\") pod \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\" (UID: \"7d022749-c77d-43f1-97fe-21f6a0cbfa2b\") "
Sep 14 03:11:58.921534 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.921395    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7d022749-c77d-43f1-97fe-21f6a0cbfa2b" (UID: "7d022749-c77d-43f1-97fe-21f6a0cbfa2b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:11:58.921534 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.921415    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7d022749-c77d-43f1-97fe-21f6a0cbfa2b" (UID: "7d022749-c77d-43f1-97fe-21f6a0cbfa2b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:11:58.921753 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.921537    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7d022749-c77d-43f1-97fe-21f6a0cbfa2b" (UID: "7d022749-c77d-43f1-97fe-21f6a0cbfa2b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:11:58.921753 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.921620    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-data" (OuterVolumeSpecName: "istio-data") pod "7d022749-c77d-43f1-97fe-21f6a0cbfa2b" (UID: "7d022749-c77d-43f1-97fe-21f6a0cbfa2b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:11:58.922099 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.921859    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:11:58.922099 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.921887    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:11:58.922099 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.921900    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:11:58.922099 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.921912    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:11:58.922099 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.921925    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:11:58.923635 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.923610    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7d022749-c77d-43f1-97fe-21f6a0cbfa2b" (UID: "7d022749-c77d-43f1-97fe-21f6a0cbfa2b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:11:58.923635 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.923617    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7d022749-c77d-43f1-97fe-21f6a0cbfa2b" (UID: "7d022749-c77d-43f1-97fe-21f6a0cbfa2b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:11:58.923635 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.923625    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-kube-api-access-fn8n4" (OuterVolumeSpecName: "kube-api-access-fn8n4") pod "7d022749-c77d-43f1-97fe-21f6a0cbfa2b" (UID: "7d022749-c77d-43f1-97fe-21f6a0cbfa2b"). InnerVolumeSpecName "kube-api-access-fn8n4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:11:58.924076 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:58.924061    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-token" (OuterVolumeSpecName: "istio-token") pod "7d022749-c77d-43f1-97fe-21f6a0cbfa2b" (UID: "7d022749-c77d-43f1-97fe-21f6a0cbfa2b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:11:59.022802 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:59.022770    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:11:59.022802 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:59.022797    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:11:59.022981 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:59.022810    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:11:59.022981 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:59.022823    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fn8n4\" (UniqueName: \"kubernetes.io/projected/7d022749-c77d-43f1-97fe-21f6a0cbfa2b-kube-api-access-fn8n4\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:11:59.516347 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:59.516308    2579 generic.go:356] "Generic (PLEG): container finished" podID="7d022749-c77d-43f1-97fe-21f6a0cbfa2b" containerID="6ac28a3f5ff46fbe0cf78f9bf72bf6d8655c3586852a827d568e721f878f5529" exitCode=0
Sep 14 03:11:59.516521 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:59.516357    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"
Sep 14 03:11:59.516521 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:59.516390    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq" event={"ID":"7d022749-c77d-43f1-97fe-21f6a0cbfa2b","Type":"ContainerDied","Data":"6ac28a3f5ff46fbe0cf78f9bf72bf6d8655c3586852a827d568e721f878f5529"}
Sep 14 03:11:59.516521 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:59.516442    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq" event={"ID":"7d022749-c77d-43f1-97fe-21f6a0cbfa2b","Type":"ContainerDied","Data":"a939808cb8696021bd96e4f400b9e7d7674c47c52162bfe6b70e090392cc8b29"}
Sep 14 03:11:59.516521 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:59.516465    2579 scope.go:117] "RemoveContainer" containerID="6ac28a3f5ff46fbe0cf78f9bf72bf6d8655c3586852a827d568e721f878f5529"
Sep 14 03:11:59.526311 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:59.526292    2579 scope.go:117] "RemoveContainer" containerID="6ac28a3f5ff46fbe0cf78f9bf72bf6d8655c3586852a827d568e721f878f5529"
Sep 14 03:11:59.526587 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:11:59.526565    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ac28a3f5ff46fbe0cf78f9bf72bf6d8655c3586852a827d568e721f878f5529\": container with ID starting with 6ac28a3f5ff46fbe0cf78f9bf72bf6d8655c3586852a827d568e721f878f5529 not found: ID does not exist" containerID="6ac28a3f5ff46fbe0cf78f9bf72bf6d8655c3586852a827d568e721f878f5529"
Sep 14 03:11:59.526680 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:59.526596    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ac28a3f5ff46fbe0cf78f9bf72bf6d8655c3586852a827d568e721f878f5529"} err="failed to get container status \"6ac28a3f5ff46fbe0cf78f9bf72bf6d8655c3586852a827d568e721f878f5529\": rpc error: code = NotFound desc = could not find container \"6ac28a3f5ff46fbe0cf78f9bf72bf6d8655c3586852a827d568e721f878f5529\": container with ID starting with 6ac28a3f5ff46fbe0cf78f9bf72bf6d8655c3586852a827d568e721f878f5529 not found: ID does not exist"
Sep 14 03:11:59.540899 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:59.540873    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"]
Sep 14 03:11:59.545897 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:59.545866    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-f16030-openshift-default-699d768b48-gqkcq"]
Sep 14 03:11:59.743934 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:11:59.743893    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d022749-c77d-43f1-97fe-21f6a0cbfa2b" path="/var/lib/kubelet/pods/7d022749-c77d-43f1-97fe-21f6a0cbfa2b/volumes"
Sep 14 03:12:06.161145 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:06.161103    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2"]
Sep 14 03:12:06.161722 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:06.161702    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d022749-c77d-43f1-97fe-21f6a0cbfa2b" containerName="istio-proxy"
Sep 14 03:12:06.161814 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:06.161725    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d022749-c77d-43f1-97fe-21f6a0cbfa2b" containerName="istio-proxy"
Sep 14 03:12:06.161878 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:06.161843    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d022749-c77d-43f1-97fe-21f6a0cbfa2b" containerName="istio-proxy"
Sep 14 03:12:06.166365 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:06.166345    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2"
Sep 14 03:12:06.168899 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:06.168881    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:12:06.169018 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:06.168977    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:12:06.173095 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:06.173071    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2"]
Sep 14 03:12:06.193719 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:06.193687    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e7f9e5a-3730-4927-a5ad-5699b25657ce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2\" (UID: \"1e7f9e5a-3730-4927-a5ad-5699b25657ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2"
Sep 14 03:12:06.295535 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:06.295490    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e7f9e5a-3730-4927-a5ad-5699b25657ce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2\" (UID: \"1e7f9e5a-3730-4927-a5ad-5699b25657ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2"
Sep 14 03:12:06.296529 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:06.296499    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e7f9e5a-3730-4927-a5ad-5699b25657ce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2\" (UID: \"1e7f9e5a-3730-4927-a5ad-5699b25657ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2"
Sep 14 03:12:06.478152 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:06.478071    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2"
Sep 14 03:12:06.778835 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:06.778810    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2"]
Sep 14 03:12:07.568067 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:07.568031    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2" event={"ID":"1e7f9e5a-3730-4927-a5ad-5699b25657ce","Type":"ContainerStarted","Data":"565c03ea30b7fe9a1b6acc8b8266a43d106b44478e1816dae96b30d74fdeab39"}
Sep 14 03:12:07.568482 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:07.568082    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2" event={"ID":"1e7f9e5a-3730-4927-a5ad-5699b25657ce","Type":"ContainerStarted","Data":"efe884b2e02ff8615f85bc4d49af59c38a9adb23f229d68ccc496dc452b54fa4"}
Sep 14 03:12:08.575225 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:08.575168    2579 generic.go:356] "Generic (PLEG): container finished" podID="1e7f9e5a-3730-4927-a5ad-5699b25657ce" containerID="565c03ea30b7fe9a1b6acc8b8266a43d106b44478e1816dae96b30d74fdeab39" exitCode=0
Sep 14 03:12:08.575634 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:08.575213    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2" event={"ID":"1e7f9e5a-3730-4927-a5ad-5699b25657ce","Type":"ContainerDied","Data":"565c03ea30b7fe9a1b6acc8b8266a43d106b44478e1816dae96b30d74fdeab39"}
Sep 14 03:12:09.745292 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:09.745267    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2"
Sep 14 03:12:09.849753 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:09.849668    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e7f9e5a-3730-4927-a5ad-5699b25657ce-maas-api-service-ca\") pod \"1e7f9e5a-3730-4927-a5ad-5699b25657ce\" (UID: \"1e7f9e5a-3730-4927-a5ad-5699b25657ce\") "
Sep 14 03:12:09.850022 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:09.849998    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e7f9e5a-3730-4927-a5ad-5699b25657ce-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1e7f9e5a-3730-4927-a5ad-5699b25657ce" (UID: "1e7f9e5a-3730-4927-a5ad-5699b25657ce"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:12:09.850102 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:09.850086    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e7f9e5a-3730-4927-a5ad-5699b25657ce-maas-api-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:12:10.589434 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:10.589408    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2"
Sep 14 03:12:10.589607 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:10.589452    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2" event={"ID":"1e7f9e5a-3730-4927-a5ad-5699b25657ce","Type":"ContainerDied","Data":"efe884b2e02ff8615f85bc4d49af59c38a9adb23f229d68ccc496dc452b54fa4"}
Sep 14 03:12:10.589607 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:10.589511    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="efe884b2e02ff8615f85bc4d49af59c38a9adb23f229d68ccc496dc452b54fa4"
Sep 14 03:12:11.527908 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:11.527087    2579 patch_prober.go:28] interesting pod/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": read tcp 10.133.0.2:54852->10.133.0.59:15021: read: connection reset by peer" start-of-body=
Sep 14 03:12:11.527908 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:11.527163    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w" podUID="265572f5-7a69-43a0-879d-62abe5b85d5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": read tcp 10.133.0.2:54852->10.133.0.59:15021: read: connection reset by peer"
Sep 14 03:12:11.597612 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:11.597586    2579 generic.go:356] "Generic (PLEG): container finished" podID="265572f5-7a69-43a0-879d-62abe5b85d5f" containerID="eb28d3ef1bd9e7427320045927585b3023de391570cc2853f6f85c4bcd17b39a" exitCode=0
Sep 14 03:12:11.597713 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:11.597630    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w" event={"ID":"265572f5-7a69-43a0-879d-62abe5b85d5f","Type":"ContainerDied","Data":"eb28d3ef1bd9e7427320045927585b3023de391570cc2853f6f85c4bcd17b39a"}
Sep 14 03:12:11.598021 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:11.598008    2579 scope.go:117] "RemoveContainer" containerID="eb28d3ef1bd9e7427320045927585b3023de391570cc2853f6f85c4bcd17b39a"
Sep 14 03:12:11.599265 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:11.599236    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:12:11.599365 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:11.599300    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:12:11.599365 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:11.599331    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:12:12.609522 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:12.609471    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w" event={"ID":"265572f5-7a69-43a0-879d-62abe5b85d5f","Type":"ContainerStarted","Data":"32b7563568c8348e44e3bded31190b46fcb4fb5cb013a1a021a25fbeaba85322"}
Sep 14 03:12:12.748370 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:12.748319    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:12:12.748600 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:12.748368    2579 patch_prober.go:28] interesting pod/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 14 03:12:12.748600 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:12.748431    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w" podUID="265572f5-7a69-43a0-879d-62abe5b85d5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 14 03:12:13.748930 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:13.748891    2579 patch_prober.go:28] interesting pod/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 14 03:12:13.749465 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:13.748950    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w" podUID="265572f5-7a69-43a0-879d-62abe5b85d5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 14 03:12:14.748521 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:14.748478    2579 patch_prober.go:28] interesting pod/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 14 03:12:14.748714 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:14.748546    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w" podUID="265572f5-7a69-43a0-879d-62abe5b85d5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 14 03:12:15.748472 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:15.748442    2579 patch_prober.go:28] interesting pod/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 14 03:12:15.748857 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:15.748497    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w" podUID="265572f5-7a69-43a0-879d-62abe5b85d5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 14 03:12:16.748633 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:16.748600    2579 patch_prober.go:28] interesting pod/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 14 03:12:16.749027 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:16.748655    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w" podUID="265572f5-7a69-43a0-879d-62abe5b85d5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 14 03:12:16.924746 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:16.924709    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"]
Sep 14 03:12:16.930620 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:16.927988    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l" podUID="6776e241-6255-4ddf-aa9a-0fdb133a5421" containerName="payload-processing" containerID="cri-o://3748ef4c64e2ee34051738ae2d04b3efbcb617276f94763145f80f60f24cc5b9" gracePeriod=30
Sep 14 03:12:16.930620 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:16.930606    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"]
Sep 14 03:12:16.931403 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:16.930859    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx" podUID="e5ce29f4-0f10-47d9-93ef-b306283df152" containerName="payload-pre-processing" containerID="cri-o://b0a89ff085c36f13e7b93a7911ad1f44268b552e772425e56a77975290f061f6" gracePeriod=30
Sep 14 03:12:17.229229 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.212028    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2"]
Sep 14 03:12:17.232464 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.232431    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4eb26b-tzkm2"]
Sep 14 03:12:17.266811 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.266750    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"
Sep 14 03:12:17.288131 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.288102    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"
Sep 14 03:12:17.433623 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.433587    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gnlq\" (UniqueName: \"kubernetes.io/projected/e5ce29f4-0f10-47d9-93ef-b306283df152-kube-api-access-5gnlq\") pod \"e5ce29f4-0f10-47d9-93ef-b306283df152\" (UID: \"e5ce29f4-0f10-47d9-93ef-b306283df152\") "
Sep 14 03:12:17.433863 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.433649    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5ce29f4-0f10-47d9-93ef-b306283df152-plugins-config-volume\") pod \"e5ce29f4-0f10-47d9-93ef-b306283df152\" (UID: \"e5ce29f4-0f10-47d9-93ef-b306283df152\") "
Sep 14 03:12:17.433863 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.433764    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2x8c\" (UniqueName: \"kubernetes.io/projected/6776e241-6255-4ddf-aa9a-0fdb133a5421-kube-api-access-w2x8c\") pod \"6776e241-6255-4ddf-aa9a-0fdb133a5421\" (UID: \"6776e241-6255-4ddf-aa9a-0fdb133a5421\") "
Sep 14 03:12:17.433863 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.433829    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6776e241-6255-4ddf-aa9a-0fdb133a5421-plugins-config-volume\") pod \"6776e241-6255-4ddf-aa9a-0fdb133a5421\" (UID: \"6776e241-6255-4ddf-aa9a-0fdb133a5421\") "
Sep 14 03:12:17.434041 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.434014    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5ce29f4-0f10-47d9-93ef-b306283df152-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e5ce29f4-0f10-47d9-93ef-b306283df152" (UID: "e5ce29f4-0f10-47d9-93ef-b306283df152"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:12:17.434261 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.434237    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6776e241-6255-4ddf-aa9a-0fdb133a5421-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6776e241-6255-4ddf-aa9a-0fdb133a5421" (UID: "6776e241-6255-4ddf-aa9a-0fdb133a5421"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:12:17.434352 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.434311    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5ce29f4-0f10-47d9-93ef-b306283df152-plugins-config-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:12:17.435984 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.435961    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6776e241-6255-4ddf-aa9a-0fdb133a5421-kube-api-access-w2x8c" (OuterVolumeSpecName: "kube-api-access-w2x8c") pod "6776e241-6255-4ddf-aa9a-0fdb133a5421" (UID: "6776e241-6255-4ddf-aa9a-0fdb133a5421"). InnerVolumeSpecName "kube-api-access-w2x8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:12:17.436042 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.435983    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5ce29f4-0f10-47d9-93ef-b306283df152-kube-api-access-5gnlq" (OuterVolumeSpecName: "kube-api-access-5gnlq") pod "e5ce29f4-0f10-47d9-93ef-b306283df152" (UID: "e5ce29f4-0f10-47d9-93ef-b306283df152"). InnerVolumeSpecName "kube-api-access-5gnlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:12:17.535939 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.535829    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gnlq\" (UniqueName: \"kubernetes.io/projected/e5ce29f4-0f10-47d9-93ef-b306283df152-kube-api-access-5gnlq\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:12:17.535939 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.535875    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2x8c\" (UniqueName: \"kubernetes.io/projected/6776e241-6255-4ddf-aa9a-0fdb133a5421-kube-api-access-w2x8c\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:12:17.535939 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.535893    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6776e241-6255-4ddf-aa9a-0fdb133a5421-plugins-config-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:12:17.641281 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.641249    2579 generic.go:356] "Generic (PLEG): container finished" podID="e5ce29f4-0f10-47d9-93ef-b306283df152" containerID="b0a89ff085c36f13e7b93a7911ad1f44268b552e772425e56a77975290f061f6" exitCode=0
Sep 14 03:12:17.641459 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.641296    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"
Sep 14 03:12:17.641459 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.641340    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx" event={"ID":"e5ce29f4-0f10-47d9-93ef-b306283df152","Type":"ContainerDied","Data":"b0a89ff085c36f13e7b93a7911ad1f44268b552e772425e56a77975290f061f6"}
Sep 14 03:12:17.641459 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.641392    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx" event={"ID":"e5ce29f4-0f10-47d9-93ef-b306283df152","Type":"ContainerDied","Data":"1aec14b23a4017227a59ddecd798e61991dcacf23801b857b97bb45943fee6a4"}
Sep 14 03:12:17.641459 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.641425    2579 scope.go:117] "RemoveContainer" containerID="b0a89ff085c36f13e7b93a7911ad1f44268b552e772425e56a77975290f061f6"
Sep 14 03:12:17.642675 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.642652    2579 generic.go:356] "Generic (PLEG): container finished" podID="6776e241-6255-4ddf-aa9a-0fdb133a5421" containerID="3748ef4c64e2ee34051738ae2d04b3efbcb617276f94763145f80f60f24cc5b9" exitCode=0
Sep 14 03:12:17.642803 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.642694    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"
Sep 14 03:12:17.642803 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.642724    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l" event={"ID":"6776e241-6255-4ddf-aa9a-0fdb133a5421","Type":"ContainerDied","Data":"3748ef4c64e2ee34051738ae2d04b3efbcb617276f94763145f80f60f24cc5b9"}
Sep 14 03:12:17.642803 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.642765    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l" event={"ID":"6776e241-6255-4ddf-aa9a-0fdb133a5421","Type":"ContainerDied","Data":"5abff48cb62c3763a4f3affc8eeac6206eafe62109e1f48654ae99c077ba6c54"}
Sep 14 03:12:17.653437 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.651254    2579 scope.go:117] "RemoveContainer" containerID="b0a89ff085c36f13e7b93a7911ad1f44268b552e772425e56a77975290f061f6"
Sep 14 03:12:17.653437 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:12:17.652062    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0a89ff085c36f13e7b93a7911ad1f44268b552e772425e56a77975290f061f6\": container with ID starting with b0a89ff085c36f13e7b93a7911ad1f44268b552e772425e56a77975290f061f6 not found: ID does not exist" containerID="b0a89ff085c36f13e7b93a7911ad1f44268b552e772425e56a77975290f061f6"
Sep 14 03:12:17.653437 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.652526    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0a89ff085c36f13e7b93a7911ad1f44268b552e772425e56a77975290f061f6"} err="failed to get container status \"b0a89ff085c36f13e7b93a7911ad1f44268b552e772425e56a77975290f061f6\": rpc error: code = NotFound desc = could not find container \"b0a89ff085c36f13e7b93a7911ad1f44268b552e772425e56a77975290f061f6\": container with ID starting with b0a89ff085c36f13e7b93a7911ad1f44268b552e772425e56a77975290f061f6 not found: ID does not exist"
Sep 14 03:12:17.653437 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.652782    2579 scope.go:117] "RemoveContainer" containerID="3748ef4c64e2ee34051738ae2d04b3efbcb617276f94763145f80f60f24cc5b9"
Sep 14 03:12:17.666488 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.666468    2579 scope.go:117] "RemoveContainer" containerID="3748ef4c64e2ee34051738ae2d04b3efbcb617276f94763145f80f60f24cc5b9"
Sep 14 03:12:17.666787 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:12:17.666766    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3748ef4c64e2ee34051738ae2d04b3efbcb617276f94763145f80f60f24cc5b9\": container with ID starting with 3748ef4c64e2ee34051738ae2d04b3efbcb617276f94763145f80f60f24cc5b9 not found: ID does not exist" containerID="3748ef4c64e2ee34051738ae2d04b3efbcb617276f94763145f80f60f24cc5b9"
Sep 14 03:12:17.666837 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.666795    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3748ef4c64e2ee34051738ae2d04b3efbcb617276f94763145f80f60f24cc5b9"} err="failed to get container status \"3748ef4c64e2ee34051738ae2d04b3efbcb617276f94763145f80f60f24cc5b9\": rpc error: code = NotFound desc = could not find container \"3748ef4c64e2ee34051738ae2d04b3efbcb617276f94763145f80f60f24cc5b9\": container with ID starting with 3748ef4c64e2ee34051738ae2d04b3efbcb617276f94763145f80f60f24cc5b9 not found: ID does not exist"
Sep 14 03:12:17.673395 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.673373    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"]
Sep 14 03:12:17.681366 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.681346    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4eb26b-648878bc46-7bsnx"]
Sep 14 03:12:17.694984 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.694963    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"]
Sep 14 03:12:17.700649 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.700622    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4eb26b-76b85b788f-hn58l"]
Sep 14 03:12:17.745262 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.745233    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e7f9e5a-3730-4927-a5ad-5699b25657ce" path="/var/lib/kubelet/pods/1e7f9e5a-3730-4927-a5ad-5699b25657ce/volumes"
Sep 14 03:12:17.745722 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.745694    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6776e241-6255-4ddf-aa9a-0fdb133a5421" path="/var/lib/kubelet/pods/6776e241-6255-4ddf-aa9a-0fdb133a5421/volumes"
Sep 14 03:12:17.746164 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.746146    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5ce29f4-0f10-47d9-93ef-b306283df152" path="/var/lib/kubelet/pods/e5ce29f4-0f10-47d9-93ef-b306283df152/volumes"
Sep 14 03:12:17.752310 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.752290    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:12:17.752709 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.752577    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:12:17.753527 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:17.753508    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:12:20.697312 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:12:20.697277    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6776e241_6255_4ddf_aa9a_0fdb133a5421.slice\": RecentStats: unable to find data in memory cache]"
Sep 14 03:12:21.029966 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:12:21.029933    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6776e241_6255_4ddf_aa9a_0fdb133a5421.slice\": RecentStats: unable to find data in memory cache]"
Sep 14 03:12:24.019059 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:24.019029    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"]
Sep 14 03:12:24.019537 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:24.019306    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj" podUID="f96c6625-2303-4192-91b5-69a588a947a0" containerName="istio-proxy" containerID="cri-o://423416f2a73e8a872615a8a30e2e4c60782c6e6be35054957bcca142b24b76fb" gracePeriod=30
Sep 14 03:12:24.949931 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:24.949898    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"]
Sep 14 03:12:24.950354 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:24.950336    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6776e241-6255-4ddf-aa9a-0fdb133a5421" containerName="payload-processing"
Sep 14 03:12:24.950354 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:24.950353    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6776e241-6255-4ddf-aa9a-0fdb133a5421" containerName="payload-processing"
Sep 14 03:12:24.950514 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:24.950372    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5ce29f4-0f10-47d9-93ef-b306283df152" containerName="payload-pre-processing"
Sep 14 03:12:24.950514 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:24.950378    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5ce29f4-0f10-47d9-93ef-b306283df152" containerName="payload-pre-processing"
Sep 14 03:12:24.950514 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:24.950385    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e7f9e5a-3730-4927-a5ad-5699b25657ce" containerName="revoke-keys"
Sep 14 03:12:24.950514 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:24.950390    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e7f9e5a-3730-4927-a5ad-5699b25657ce" containerName="revoke-keys"
Sep 14 03:12:24.950514 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:24.950450    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6776e241-6255-4ddf-aa9a-0fdb133a5421" containerName="payload-processing"
Sep 14 03:12:24.950514 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:24.950459    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e7f9e5a-3730-4927-a5ad-5699b25657ce" containerName="revoke-keys"
Sep 14 03:12:24.950514 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:24.950469    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5ce29f4-0f10-47d9-93ef-b306283df152" containerName="payload-pre-processing"
Sep 14 03:12:24.955069 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:24.955052    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:24.958028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:24.958010    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-2b32aa-openshift-default-dockercfg-7smcl\""
Sep 14 03:12:24.967877 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:24.967851    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"]
Sep 14 03:12:25.110332 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.110300    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-workload-certs\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.110774 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.110343    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5da2703a-da04-4553-ad09-b76929116540-istio-token\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.110774 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.110369    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-istio-data\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.110774 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.110414    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5da2703a-da04-4553-ad09-b76929116540-istio-podinfo\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.110774 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.110483    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5da2703a-da04-4553-ad09-b76929116540-istiod-ca-cert\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.110774 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.110519    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-istio-envoy\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.110774 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.110542    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-workload-socket\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.110774 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.110657    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-credential-socket\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.110774 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.110707    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dmcs\" (UniqueName: \"kubernetes.io/projected/5da2703a-da04-4553-ad09-b76929116540-kube-api-access-8dmcs\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.211499 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.211404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5da2703a-da04-4553-ad09-b76929116540-istiod-ca-cert\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.211499 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.211462    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-istio-envoy\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.211499 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.211494    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-workload-socket\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.211784 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.211560    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-credential-socket\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.211784 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.211598    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dmcs\" (UniqueName: \"kubernetes.io/projected/5da2703a-da04-4553-ad09-b76929116540-kube-api-access-8dmcs\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.211784 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.211631    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-workload-certs\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.211784 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.211664    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5da2703a-da04-4553-ad09-b76929116540-istio-token\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.211784 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.211703    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-istio-data\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.211784 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.211728    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5da2703a-da04-4553-ad09-b76929116540-istio-podinfo\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.212093 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.211884    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-workload-socket\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.212093 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.211972    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-workload-certs\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.212239 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.212112    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-credential-socket\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.212239 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.212132    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-istio-data\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.212239 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.212161    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5da2703a-da04-4553-ad09-b76929116540-istiod-ca-cert\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.213876 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.213840    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-istio-envoy\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.214188 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.214158    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5da2703a-da04-4553-ad09-b76929116540-istio-podinfo\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.220768 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.220745    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5da2703a-da04-4553-ad09-b76929116540-istio-token\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.221158 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.221139    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dmcs\" (UniqueName: \"kubernetes.io/projected/5da2703a-da04-4553-ad09-b76929116540-kube-api-access-8dmcs\") pod \"e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") " pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.268956 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.268914    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:25.547360 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.547332    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"]
Sep 14 03:12:25.549288 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:12:25.549247    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da2703a_da04_4553_ad09_b76929116540.slice/crio-e4b6a7718340008b118111121ad0c557df2cd8eadecf38687faeb1bc559de8c8 WatchSource:0}: Error finding container e4b6a7718340008b118111121ad0c557df2cd8eadecf38687faeb1bc559de8c8: Status 404 returned error can't find the container with id e4b6a7718340008b118111121ad0c557df2cd8eadecf38687faeb1bc559de8c8
Sep 14 03:12:25.551819 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.551787    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:12:25.551940 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.551862    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:12:25.551940 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.551909    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:12:25.686246 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.686208    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm" event={"ID":"5da2703a-da04-4553-ad09-b76929116540","Type":"ContainerStarted","Data":"d407408ab6137c8fc884808010aafab1cf97483928af25a3175a066b7587dcd5"}
Sep 14 03:12:25.686440 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:25.686278    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm" event={"ID":"5da2703a-da04-4553-ad09-b76929116540","Type":"ContainerStarted","Data":"e4b6a7718340008b118111121ad0c557df2cd8eadecf38687faeb1bc559de8c8"}
Sep 14 03:12:26.270122 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:26.270089    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:26.275305 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:26.275282    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:26.298107 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:26.298049    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm" podStartSLOduration=2.298029075 podStartE2EDuration="2.298029075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:12:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:12:25.712538285 +0000 UTC m=+1208.498600224" watchObservedRunningTime="2026-09-14 03:12:26.298029075 +0000 UTC m=+1209.084090989"
Sep 14 03:12:26.691558 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:26.691466    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:26.692451 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:26.692429    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:12:28.651621 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:28.651585    2579 patch_prober.go:28] interesting pod/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 03:12:28.652007 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:28.651643    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj" podUID="f96c6625-2303-4192-91b5-69a588a947a0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 03:12:29.282167 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.282107    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:12:29.351665 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.351636    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f96c6625-2303-4192-91b5-69a588a947a0-istio-token\") pod \"f96c6625-2303-4192-91b5-69a588a947a0\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") "
Sep 14 03:12:29.351665 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.351669    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f96c6625-2303-4192-91b5-69a588a947a0-istiod-ca-cert\") pod \"f96c6625-2303-4192-91b5-69a588a947a0\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") "
Sep 14 03:12:29.351870 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.351721    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f96c6625-2303-4192-91b5-69a588a947a0-istio-podinfo\") pod \"f96c6625-2303-4192-91b5-69a588a947a0\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") "
Sep 14 03:12:29.351870 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.351749    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-credential-socket\") pod \"f96c6625-2303-4192-91b5-69a588a947a0\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") "
Sep 14 03:12:29.351870 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.351779    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-workload-socket\") pod \"f96c6625-2303-4192-91b5-69a588a947a0\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") "
Sep 14 03:12:29.351870 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.351814    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-istio-data\") pod \"f96c6625-2303-4192-91b5-69a588a947a0\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") "
Sep 14 03:12:29.351870 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.351850    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cx77f\" (UniqueName: \"kubernetes.io/projected/f96c6625-2303-4192-91b5-69a588a947a0-kube-api-access-cx77f\") pod \"f96c6625-2303-4192-91b5-69a588a947a0\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") "
Sep 14 03:12:29.352118 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.351876    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-workload-certs\") pod \"f96c6625-2303-4192-91b5-69a588a947a0\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") "
Sep 14 03:12:29.352118 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.351899    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-istio-envoy\") pod \"f96c6625-2303-4192-91b5-69a588a947a0\" (UID: \"f96c6625-2303-4192-91b5-69a588a947a0\") "
Sep 14 03:12:29.352118 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.352025    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f96c6625-2303-4192-91b5-69a588a947a0" (UID: "f96c6625-2303-4192-91b5-69a588a947a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:12:29.352314 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.352135    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f96c6625-2303-4192-91b5-69a588a947a0" (UID: "f96c6625-2303-4192-91b5-69a588a947a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:12:29.352314 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.352150    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-istio-data" (OuterVolumeSpecName: "istio-data") pod "f96c6625-2303-4192-91b5-69a588a947a0" (UID: "f96c6625-2303-4192-91b5-69a588a947a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:12:29.352314 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.352226    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f96c6625-2303-4192-91b5-69a588a947a0" (UID: "f96c6625-2303-4192-91b5-69a588a947a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:12:29.352444 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.352420    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:12:29.352444 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.352424    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f96c6625-2303-4192-91b5-69a588a947a0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f96c6625-2303-4192-91b5-69a588a947a0" (UID: "f96c6625-2303-4192-91b5-69a588a947a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:12:29.352552 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.352442    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:12:29.352552 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.352457    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:12:29.352552 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.352470    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:12:29.354333 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.354304    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f96c6625-2303-4192-91b5-69a588a947a0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f96c6625-2303-4192-91b5-69a588a947a0" (UID: "f96c6625-2303-4192-91b5-69a588a947a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:12:29.354484 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.354464    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f96c6625-2303-4192-91b5-69a588a947a0" (UID: "f96c6625-2303-4192-91b5-69a588a947a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:12:29.354551 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.354537    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f96c6625-2303-4192-91b5-69a588a947a0-istio-token" (OuterVolumeSpecName: "istio-token") pod "f96c6625-2303-4192-91b5-69a588a947a0" (UID: "f96c6625-2303-4192-91b5-69a588a947a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:12:29.354612 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.354581    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f96c6625-2303-4192-91b5-69a588a947a0-kube-api-access-cx77f" (OuterVolumeSpecName: "kube-api-access-cx77f") pod "f96c6625-2303-4192-91b5-69a588a947a0" (UID: "f96c6625-2303-4192-91b5-69a588a947a0"). InnerVolumeSpecName "kube-api-access-cx77f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:12:29.453310 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.453279    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f96c6625-2303-4192-91b5-69a588a947a0-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:12:29.453310 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.453304    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f96c6625-2303-4192-91b5-69a588a947a0-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:12:29.453310 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.453312    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f96c6625-2303-4192-91b5-69a588a947a0-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:12:29.453525 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.453321    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx77f\" (UniqueName: \"kubernetes.io/projected/f96c6625-2303-4192-91b5-69a588a947a0-kube-api-access-cx77f\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:12:29.453525 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.453329    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f96c6625-2303-4192-91b5-69a588a947a0-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:12:29.709613 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.709528    2579 generic.go:356] "Generic (PLEG): container finished" podID="f96c6625-2303-4192-91b5-69a588a947a0" containerID="423416f2a73e8a872615a8a30e2e4c60782c6e6be35054957bcca142b24b76fb" exitCode=0
Sep 14 03:12:29.709613 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.709573    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"
Sep 14 03:12:29.709613 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.709604    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj" event={"ID":"f96c6625-2303-4192-91b5-69a588a947a0","Type":"ContainerDied","Data":"423416f2a73e8a872615a8a30e2e4c60782c6e6be35054957bcca142b24b76fb"}
Sep 14 03:12:29.710126 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.709654    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj" event={"ID":"f96c6625-2303-4192-91b5-69a588a947a0","Type":"ContainerDied","Data":"280f638359fb5bc6f6557ea86e9477f8b6607671aee36b7349e20f84913e90c5"}
Sep 14 03:12:29.710126 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.709676    2579 scope.go:117] "RemoveContainer" containerID="423416f2a73e8a872615a8a30e2e4c60782c6e6be35054957bcca142b24b76fb"
Sep 14 03:12:29.719297 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.719279    2579 scope.go:117] "RemoveContainer" containerID="4b2e5a55901021eff3664a69293725d5f2df4bfc16ea7e945c2dc46b4c693863"
Sep 14 03:12:29.727392 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.727376    2579 scope.go:117] "RemoveContainer" containerID="423416f2a73e8a872615a8a30e2e4c60782c6e6be35054957bcca142b24b76fb"
Sep 14 03:12:29.727643 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:12:29.727623    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"423416f2a73e8a872615a8a30e2e4c60782c6e6be35054957bcca142b24b76fb\": container with ID starting with 423416f2a73e8a872615a8a30e2e4c60782c6e6be35054957bcca142b24b76fb not found: ID does not exist" containerID="423416f2a73e8a872615a8a30e2e4c60782c6e6be35054957bcca142b24b76fb"
Sep 14 03:12:29.727720 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.727655    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423416f2a73e8a872615a8a30e2e4c60782c6e6be35054957bcca142b24b76fb"} err="failed to get container status \"423416f2a73e8a872615a8a30e2e4c60782c6e6be35054957bcca142b24b76fb\": rpc error: code = NotFound desc = could not find container \"423416f2a73e8a872615a8a30e2e4c60782c6e6be35054957bcca142b24b76fb\": container with ID starting with 423416f2a73e8a872615a8a30e2e4c60782c6e6be35054957bcca142b24b76fb not found: ID does not exist"
Sep 14 03:12:29.727720 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.727679    2579 scope.go:117] "RemoveContainer" containerID="4b2e5a55901021eff3664a69293725d5f2df4bfc16ea7e945c2dc46b4c693863"
Sep 14 03:12:29.727930 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:12:29.727912    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b2e5a55901021eff3664a69293725d5f2df4bfc16ea7e945c2dc46b4c693863\": container with ID starting with 4b2e5a55901021eff3664a69293725d5f2df4bfc16ea7e945c2dc46b4c693863 not found: ID does not exist" containerID="4b2e5a55901021eff3664a69293725d5f2df4bfc16ea7e945c2dc46b4c693863"
Sep 14 03:12:29.727966 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.727936    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b2e5a55901021eff3664a69293725d5f2df4bfc16ea7e945c2dc46b4c693863"} err="failed to get container status \"4b2e5a55901021eff3664a69293725d5f2df4bfc16ea7e945c2dc46b4c693863\": rpc error: code = NotFound desc = could not find container \"4b2e5a55901021eff3664a69293725d5f2df4bfc16ea7e945c2dc46b4c693863\": container with ID starting with 4b2e5a55901021eff3664a69293725d5f2df4bfc16ea7e945c2dc46b4c693863 not found: ID does not exist"
Sep 14 03:12:29.735950 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.735931    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"]
Sep 14 03:12:29.743920 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:29.743899    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-4eb26b-openshift-default-669575d694-vmqnj"]
Sep 14 03:12:31.082215 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:12:31.082184    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6776e241_6255_4ddf_aa9a_0fdb133a5421.slice\": RecentStats: unable to find data in memory cache]"
Sep 14 03:12:31.661092 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.661061    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"]
Sep 14 03:12:31.661476 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.661460    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f96c6625-2303-4192-91b5-69a588a947a0" containerName="istio-proxy"
Sep 14 03:12:31.661476 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.661475    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f96c6625-2303-4192-91b5-69a588a947a0" containerName="istio-proxy"
Sep 14 03:12:31.661636 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.661547    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f96c6625-2303-4192-91b5-69a588a947a0" containerName="istio-proxy"
Sep 14 03:12:31.661636 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.661555    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f96c6625-2303-4192-91b5-69a588a947a0" containerName="istio-proxy"
Sep 14 03:12:31.666030 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.666008    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:12:31.668741 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.668721    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-2b32aa-serving-cert\""
Sep 14 03:12:31.669798 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.669779    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jf78h\""
Sep 14 03:12:31.669939 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.669907    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 03:12:31.670134 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.669819    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:12:31.670280 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.670261    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 03:12:31.677058 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.677033    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"]
Sep 14 03:12:31.744511 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.744483    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f96c6625-2303-4192-91b5-69a588a947a0" path="/var/lib/kubelet/pods/f96c6625-2303-4192-91b5-69a588a947a0/volumes"
Sep 14 03:12:31.782416 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.782378    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4bkp\" (UniqueName: \"kubernetes.io/projected/cc167e10-f611-4ff9-a565-9359ee291d76-kube-api-access-l4bkp\") pod \"maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s\" (UID: \"cc167e10-f611-4ff9-a565-9359ee291d76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:12:31.782777 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.782750    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc167e10-f611-4ff9-a565-9359ee291d76-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s\" (UID: \"cc167e10-f611-4ff9-a565-9359ee291d76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:12:31.783013 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.782990    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc167e10-f611-4ff9-a565-9359ee291d76-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s\" (UID: \"cc167e10-f611-4ff9-a565-9359ee291d76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:12:31.783225 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.783198    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc167e10-f611-4ff9-a565-9359ee291d76-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s\" (UID: \"cc167e10-f611-4ff9-a565-9359ee291d76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:12:31.884763 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.884731    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4bkp\" (UniqueName: \"kubernetes.io/projected/cc167e10-f611-4ff9-a565-9359ee291d76-kube-api-access-l4bkp\") pod \"maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s\" (UID: \"cc167e10-f611-4ff9-a565-9359ee291d76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:12:31.884903 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.884771    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc167e10-f611-4ff9-a565-9359ee291d76-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s\" (UID: \"cc167e10-f611-4ff9-a565-9359ee291d76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:12:31.884903 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.884815    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc167e10-f611-4ff9-a565-9359ee291d76-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s\" (UID: \"cc167e10-f611-4ff9-a565-9359ee291d76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:12:31.884903 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.884852    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc167e10-f611-4ff9-a565-9359ee291d76-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s\" (UID: \"cc167e10-f611-4ff9-a565-9359ee291d76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:12:31.885509 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.885488    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc167e10-f611-4ff9-a565-9359ee291d76-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s\" (UID: \"cc167e10-f611-4ff9-a565-9359ee291d76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:12:31.887480 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.887455    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc167e10-f611-4ff9-a565-9359ee291d76-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s\" (UID: \"cc167e10-f611-4ff9-a565-9359ee291d76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:12:31.887480 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.887473    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc167e10-f611-4ff9-a565-9359ee291d76-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s\" (UID: \"cc167e10-f611-4ff9-a565-9359ee291d76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:12:31.895741 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.895721    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4bkp\" (UniqueName: \"kubernetes.io/projected/cc167e10-f611-4ff9-a565-9359ee291d76-kube-api-access-l4bkp\") pod \"maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s\" (UID: \"cc167e10-f611-4ff9-a565-9359ee291d76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:12:31.978521 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:31.978459    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:12:32.268681 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.268658    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"]
Sep 14 03:12:32.505018 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.504982    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"]
Sep 14 03:12:32.505981 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.505964    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f96c6625-2303-4192-91b5-69a588a947a0" containerName="istio-proxy"
Sep 14 03:12:32.506115 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.506104    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f96c6625-2303-4192-91b5-69a588a947a0" containerName="istio-proxy"
Sep 14 03:12:32.511740 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.511715    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"
Sep 14 03:12:32.515965 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.515940    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-2b32aa-dockercfg-4cwkg\""
Sep 14 03:12:32.516684 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.516408    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-2b32aa\""
Sep 14 03:12:32.532216 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.532185    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"]
Sep 14 03:12:32.591906 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.591879    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqv8l\" (UniqueName: \"kubernetes.io/projected/ee8688c3-0edf-42bb-8157-c8100d5c53c9-kube-api-access-hqv8l\") pod \"payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl\" (UID: \"ee8688c3-0edf-42bb-8157-c8100d5c53c9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"
Sep 14 03:12:32.592077 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.591913    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee8688c3-0edf-42bb-8157-c8100d5c53c9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl\" (UID: \"ee8688c3-0edf-42bb-8157-c8100d5c53c9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"
Sep 14 03:12:32.692808 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.692782    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqv8l\" (UniqueName: \"kubernetes.io/projected/ee8688c3-0edf-42bb-8157-c8100d5c53c9-kube-api-access-hqv8l\") pod \"payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl\" (UID: \"ee8688c3-0edf-42bb-8157-c8100d5c53c9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"
Sep 14 03:12:32.692948 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.692816    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee8688c3-0edf-42bb-8157-c8100d5c53c9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl\" (UID: \"ee8688c3-0edf-42bb-8157-c8100d5c53c9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"
Sep 14 03:12:32.693588 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.693567    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee8688c3-0edf-42bb-8157-c8100d5c53c9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl\" (UID: \"ee8688c3-0edf-42bb-8157-c8100d5c53c9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"
Sep 14 03:12:32.703598 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.703578    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqv8l\" (UniqueName: \"kubernetes.io/projected/ee8688c3-0edf-42bb-8157-c8100d5c53c9-kube-api-access-hqv8l\") pod \"payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl\" (UID: \"ee8688c3-0edf-42bb-8157-c8100d5c53c9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"
Sep 14 03:12:32.727702 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.727678    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s" event={"ID":"cc167e10-f611-4ff9-a565-9359ee291d76","Type":"ContainerStarted","Data":"30a4e3146357e468a9fe5000a65ab53e855a2a2b82bf8ecb14fdc074d9af1e4f"}
Sep 14 03:12:32.727792 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.727722    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s" event={"ID":"cc167e10-f611-4ff9-a565-9359ee291d76","Type":"ContainerStarted","Data":"a8aa6a19fe9df49ad9d30dadd12fd7a5a5b1dc20928c03168b536a07fb22c8bb"}
Sep 14 03:12:32.727855 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.727810    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:12:32.753276 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.753232    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s" podStartSLOduration=1.4014762840000001 podStartE2EDuration="1.753218011s" totalImagesPullingTime="351.741727ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:12:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:12:32.749747982 +0000 UTC m=+1215.535809895" watchObservedRunningTime="2026-09-14 03:12:32.753218011 +0000 UTC m=+1215.539279922"
Sep 14 03:12:32.828912 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:32.828826    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"
Sep 14 03:12:33.117417 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:33.117387    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"]
Sep 14 03:12:33.119417 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:12:33.119396    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee8688c3_0edf_42bb_8157_c8100d5c53c9.slice/crio-66aa8c33064bf25f1c7882ac745a1934c95e79e37da3b65bc7fa2222143a4166 WatchSource:0}: Error finding container 66aa8c33064bf25f1c7882ac745a1934c95e79e37da3b65bc7fa2222143a4166: Status 404 returned error can't find the container with id 66aa8c33064bf25f1c7882ac745a1934c95e79e37da3b65bc7fa2222143a4166
Sep 14 03:12:33.734827 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:33.734792    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl" event={"ID":"ee8688c3-0edf-42bb-8157-c8100d5c53c9","Type":"ContainerStarted","Data":"7fccde9b8aff9c5d6a1906b0f4368ae0806fd1376f6e45c92cf70f98c5ee81e5"}
Sep 14 03:12:33.735339 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:33.734845    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl" event={"ID":"ee8688c3-0edf-42bb-8157-c8100d5c53c9","Type":"ContainerStarted","Data":"66aa8c33064bf25f1c7882ac745a1934c95e79e37da3b65bc7fa2222143a4166"}
Sep 14 03:12:33.735339 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:33.734952    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"
Sep 14 03:12:33.755243 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:33.755200    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl" podStartSLOduration=1.755169204 podStartE2EDuration="1.755169204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:12:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:12:33.753421359 +0000 UTC m=+1216.539483273" watchObservedRunningTime="2026-09-14 03:12:33.755169204 +0000 UTC m=+1216.541231116"
Sep 14 03:12:35.679546 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:12:35.679512    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6776e241_6255_4ddf_aa9a_0fdb133a5421.slice\": RecentStats: unable to find data in memory cache]"
Sep 14 03:12:38.740830 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:38.740801    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:12:41.092131 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:12:41.092095    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6776e241_6255_4ddf_aa9a_0fdb133a5421.slice\": RecentStats: unable to find data in memory cache]"
Sep 14 03:12:44.743132 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:44.743099    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"
Sep 14 03:12:50.712364 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:12:50.712329    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6776e241_6255_4ddf_aa9a_0fdb133a5421.slice\": RecentStats: unable to find data in memory cache]"
Sep 14 03:12:51.101983 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:12:51.101950    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6776e241_6255_4ddf_aa9a_0fdb133a5421.slice\": RecentStats: unable to find data in memory cache]"
Sep 14 03:12:54.002205 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:12:54.000776    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6776e241_6255_4ddf_aa9a_0fdb133a5421.slice\": RecentStats: unable to find data in memory cache]"
Sep 14 03:12:54.012205 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:12:54.009117    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6776e241_6255_4ddf_aa9a_0fdb133a5421.slice\": RecentStats: unable to find data in memory cache]"
Sep 14 03:12:58.265763 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.265716    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"]
Sep 14 03:12:58.271216 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.271195    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.274110 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.274087    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-177593-openshift-default-dockercfg-jc5p7\""
Sep 14 03:12:58.282161 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.282133    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"]
Sep 14 03:12:58.355886 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.355796    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d18248d-5877-4afd-881a-21f6441993e1-istio-token\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.355886 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.355880    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d18248d-5877-4afd-881a-21f6441993e1-istio-podinfo\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.356231 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.355927    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-workload-certs\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.356231 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.355954    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d18248d-5877-4afd-881a-21f6441993e1-istiod-ca-cert\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.356231 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.355979    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-workload-socket\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.356231 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.356026    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-istio-data\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.356231 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.356067    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-istio-envoy\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.356231 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.356136    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-credential-socket\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.356231 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.356215    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbvvn\" (UniqueName: \"kubernetes.io/projected/0d18248d-5877-4afd-881a-21f6441993e1-kube-api-access-rbvvn\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.456959 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.456920    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-credential-socket\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.457157 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.456967    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbvvn\" (UniqueName: \"kubernetes.io/projected/0d18248d-5877-4afd-881a-21f6441993e1-kube-api-access-rbvvn\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.457157 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.457012    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d18248d-5877-4afd-881a-21f6441993e1-istio-token\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.457157 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.457041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d18248d-5877-4afd-881a-21f6441993e1-istio-podinfo\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.457157 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.457081    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-workload-certs\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.457157 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.457110    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d18248d-5877-4afd-881a-21f6441993e1-istiod-ca-cert\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.457157 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.457136    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-workload-socket\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.457525 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.457212    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-istio-data\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.457525 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.457251    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-istio-envoy\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.457525 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.457383    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-credential-socket\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.457686 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.457645    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-istio-data\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.457686 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.457666    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-workload-socket\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.457796 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.457752    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-workload-certs\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.457923 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.457901    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d18248d-5877-4afd-881a-21f6441993e1-istiod-ca-cert\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.459891 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.459870    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-istio-envoy\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.460211 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.460167    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d18248d-5877-4afd-881a-21f6441993e1-istio-podinfo\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.465325 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.465303    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbvvn\" (UniqueName: \"kubernetes.io/projected/0d18248d-5877-4afd-881a-21f6441993e1-kube-api-access-rbvvn\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.465481 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.465460    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d18248d-5877-4afd-881a-21f6441993e1-istio-token\") pod \"e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") " pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.584954 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.584869    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:12:58.868687 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.868552    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"]
Sep 14 03:12:58.871711 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:12:58.871673    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d18248d_5877_4afd_881a_21f6441993e1.slice/crio-50a5a5f9885270c0319171005bbe2e81bbb72942a71db9e53f656f7b4fc688a0 WatchSource:0}: Error finding container 50a5a5f9885270c0319171005bbe2e81bbb72942a71db9e53f656f7b4fc688a0: Status 404 returned error can't find the container with id 50a5a5f9885270c0319171005bbe2e81bbb72942a71db9e53f656f7b4fc688a0
Sep 14 03:12:58.874267 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.874231    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:12:58.874345 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.874317    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:12:58.874398 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:58.874349    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:12:59.881766 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:59.881728    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t" event={"ID":"0d18248d-5877-4afd-881a-21f6441993e1","Type":"ContainerStarted","Data":"9440856ddb3a1a3b4eba241ff4f696a53c3930921561e03392f80f729d2d1c0e"}
Sep 14 03:12:59.882271 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:59.881778    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t" event={"ID":"0d18248d-5877-4afd-881a-21f6441993e1","Type":"ContainerStarted","Data":"50a5a5f9885270c0319171005bbe2e81bbb72942a71db9e53f656f7b4fc688a0"}
Sep 14 03:12:59.905095 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:12:59.905044    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t" podStartSLOduration=1.905030144 podStartE2EDuration="1.905030144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:12:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:12:59.902720797 +0000 UTC m=+1242.688782724" watchObservedRunningTime="2026-09-14 03:12:59.905030144 +0000 UTC m=+1242.691092056"
Sep 14 03:13:00.585254 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:00.585219    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:13:00.590319 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:00.590299    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:13:00.888981 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:00.888890    2579 generic.go:356] "Generic (PLEG): container finished" podID="5da2703a-da04-4553-ad09-b76929116540" containerID="d407408ab6137c8fc884808010aafab1cf97483928af25a3175a066b7587dcd5" exitCode=0
Sep 14 03:13:00.888981 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:00.888966    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm" event={"ID":"5da2703a-da04-4553-ad09-b76929116540","Type":"ContainerDied","Data":"d407408ab6137c8fc884808010aafab1cf97483928af25a3175a066b7587dcd5"}
Sep 14 03:13:00.889519 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:00.889160    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:13:00.889580 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:00.889525    2579 scope.go:117] "RemoveContainer" containerID="d407408ab6137c8fc884808010aafab1cf97483928af25a3175a066b7587dcd5"
Sep 14 03:13:00.890439 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:00.890118    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:13:00.890728 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:00.890704    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:13:00.890776 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:00.890764    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:13:00.890812 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:00.890796    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:13:01.152794 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:13:01.152708    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6776e241_6255_4ddf_aa9a_0fdb133a5421.slice\": RecentStats: unable to find data in memory cache]"
Sep 14 03:13:01.900514 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:01.900478    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm" event={"ID":"5da2703a-da04-4553-ad09-b76929116540","Type":"ContainerStarted","Data":"d5b979eb1fa3270356c5e2f4d820134a8c43ccf8d62f30728177d0510f5b734c"}
Sep 14 03:13:02.269425 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:02.269392    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:13:02.269590 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:02.269564    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 14 03:13:02.269635 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:02.269614    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm" podUID="5da2703a-da04-4553-ad09-b76929116540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 14 03:13:03.269253 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:03.269215    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 14 03:13:03.269639 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:03.269272    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm" podUID="5da2703a-da04-4553-ad09-b76929116540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 14 03:13:04.270041 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:04.270005    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 14 03:13:04.270451 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:04.270082    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm" podUID="5da2703a-da04-4553-ad09-b76929116540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 14 03:13:05.269290 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:05.269249    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 14 03:13:05.269505 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:05.269312    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm" podUID="5da2703a-da04-4553-ad09-b76929116540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 14 03:13:05.670792 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:13:05.670699    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6776e241_6255_4ddf_aa9a_0fdb133a5421.slice\": RecentStats: unable to find data in memory cache]"
Sep 14 03:13:06.270308 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:06.270270    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 14 03:13:06.270481 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:06.270343    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm" podUID="5da2703a-da04-4553-ad09-b76929116540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 14 03:13:07.273627 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:07.273596    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:13:07.274025 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:07.273892    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:13:07.274678 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:07.274661    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:13:11.202751 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:13:11.202673    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6776e241_6255_4ddf_aa9a_0fdb133a5421.slice\": RecentStats: unable to find data in memory cache]"
Sep 14 03:13:17.860185 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:17.860152    2579 scope.go:117] "RemoveContainer" containerID="88ac0d9ce27b1e677ab4b378d507c27842009a7b19e9b70ce8cbb756317a053d"
Sep 14 03:13:17.868966 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:17.868941    2579 scope.go:117] "RemoveContainer" containerID="ceebfb8b10525ee0c84afc893e039afac02fca69929e46f88ce38692a3bf3456"
Sep 14 03:13:38.107990 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:38.107959    2579 generic.go:356] "Generic (PLEG): container finished" podID="0d18248d-5877-4afd-881a-21f6441993e1" containerID="9440856ddb3a1a3b4eba241ff4f696a53c3930921561e03392f80f729d2d1c0e" exitCode=0
Sep 14 03:13:38.108434 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:38.108031    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t" event={"ID":"0d18248d-5877-4afd-881a-21f6441993e1","Type":"ContainerDied","Data":"9440856ddb3a1a3b4eba241ff4f696a53c3930921561e03392f80f729d2d1c0e"}
Sep 14 03:13:38.108434 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:38.108423    2579 scope.go:117] "RemoveContainer" containerID="9440856ddb3a1a3b4eba241ff4f696a53c3930921561e03392f80f729d2d1c0e"
Sep 14 03:13:38.109531 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:38.109504    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:13:38.109642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:38.109584    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:13:38.109642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:38.109633    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:13:39.115982 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:39.115949    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t" event={"ID":"0d18248d-5877-4afd-881a-21f6441993e1","Type":"ContainerStarted","Data":"2d422940da295c1e69909277e975787034c221f3b72fd547dfa7c5948d31661e"}
Sep 14 03:13:39.585691 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:39.585660    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:13:39.585867 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:39.585818    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 14 03:13:39.585931 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:39.585872    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t" podUID="0d18248d-5877-4afd-881a-21f6441993e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 14 03:13:40.586218 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:40.586154    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 14 03:13:40.586612 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:40.586254    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t" podUID="0d18248d-5877-4afd-881a-21f6441993e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 14 03:13:41.586275 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:41.586238    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 14 03:13:41.586706 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:41.586296    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t" podUID="0d18248d-5877-4afd-881a-21f6441993e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 14 03:13:42.585561 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:42.585528    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 14 03:13:42.585741 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:42.585585    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t" podUID="0d18248d-5877-4afd-881a-21f6441993e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 14 03:13:43.585510 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:43.585473    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 14 03:13:43.585922 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:43.585534    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t" podUID="0d18248d-5877-4afd-881a-21f6441993e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 14 03:13:44.586037 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:44.586002    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 14 03:13:44.586450 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:44.586058    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t" podUID="0d18248d-5877-4afd-881a-21f6441993e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 14 03:13:45.589870 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:45.589834    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:13:45.590272 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:45.590218    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:13:45.590803 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:45.590785    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:13:47.221616 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:47.221583    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m"]
Sep 14 03:13:47.223979 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:47.223964    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m"
Sep 14 03:13:47.227573 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:47.227542    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:13:47.236128 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:47.236107    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m"]
Sep 14 03:13:47.250398 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:47.250375    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bae37c3c-ace6-453c-a083-876a010df969-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m\" (UID: \"bae37c3c-ace6-453c-a083-876a010df969\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m"
Sep 14 03:13:47.351802 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:47.351773    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bae37c3c-ace6-453c-a083-876a010df969-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m\" (UID: \"bae37c3c-ace6-453c-a083-876a010df969\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m"
Sep 14 03:13:47.352376 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:47.352358    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bae37c3c-ace6-453c-a083-876a010df969-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m\" (UID: \"bae37c3c-ace6-453c-a083-876a010df969\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m"
Sep 14 03:13:47.535292 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:47.535259    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m"
Sep 14 03:13:47.806157 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:47.806013    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m"]
Sep 14 03:13:47.809187 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:13:47.809146    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbae37c3c_ace6_453c_a083_876a010df969.slice/crio-2b4f4a371ed8eaddbc06555760105a4a394db91dd431fd0aafdf2294632432a9 WatchSource:0}: Error finding container 2b4f4a371ed8eaddbc06555760105a4a394db91dd431fd0aafdf2294632432a9: Status 404 returned error can't find the container with id 2b4f4a371ed8eaddbc06555760105a4a394db91dd431fd0aafdf2294632432a9
Sep 14 03:13:48.166993 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:48.166966    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m" event={"ID":"bae37c3c-ace6-453c-a083-876a010df969","Type":"ContainerStarted","Data":"31b7a50275d151f8825e7e71305f00822ae081032bd4e777d73724f431d626e5"}
Sep 14 03:13:48.167106 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:48.167019    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m" event={"ID":"bae37c3c-ace6-453c-a083-876a010df969","Type":"ContainerStarted","Data":"2b4f4a371ed8eaddbc06555760105a4a394db91dd431fd0aafdf2294632432a9"}
Sep 14 03:13:48.190519 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:48.190477    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m" podStartSLOduration=1.190462395 podStartE2EDuration="1.190462395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:13:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:13:48.187533169 +0000 UTC m=+1290.973595082" watchObservedRunningTime="2026-09-14 03:13:48.190462395 +0000 UTC m=+1290.976524307"
Sep 14 03:13:49.174418 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:49.174382    2579 generic.go:356] "Generic (PLEG): container finished" podID="bae37c3c-ace6-453c-a083-876a010df969" containerID="31b7a50275d151f8825e7e71305f00822ae081032bd4e777d73724f431d626e5" exitCode=0
Sep 14 03:13:49.174790 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:49.174474    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m" event={"ID":"bae37c3c-ace6-453c-a083-876a010df969","Type":"ContainerDied","Data":"31b7a50275d151f8825e7e71305f00822ae081032bd4e777d73724f431d626e5"}
Sep 14 03:13:50.324538 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:50.324513    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m"
Sep 14 03:13:50.385734 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:50.385696    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bae37c3c-ace6-453c-a083-876a010df969-maas-api-service-ca\") pod \"bae37c3c-ace6-453c-a083-876a010df969\" (UID: \"bae37c3c-ace6-453c-a083-876a010df969\") "
Sep 14 03:13:50.386026 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:50.386005    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bae37c3c-ace6-453c-a083-876a010df969-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bae37c3c-ace6-453c-a083-876a010df969" (UID: "bae37c3c-ace6-453c-a083-876a010df969"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:13:50.487391 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:50.487320    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bae37c3c-ace6-453c-a083-876a010df969-maas-api-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:13:51.186946 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:51.186918    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m"
Sep 14 03:13:51.187116 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:51.186960    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m" event={"ID":"bae37c3c-ace6-453c-a083-876a010df969","Type":"ContainerDied","Data":"2b4f4a371ed8eaddbc06555760105a4a394db91dd431fd0aafdf2294632432a9"}
Sep 14 03:13:51.187116 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:13:51.187005    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b4f4a371ed8eaddbc06555760105a4a394db91dd431fd0aafdf2294632432a9"
Sep 14 03:14:03.380540 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:14:03.380502    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m"]
Sep 14 03:14:03.389222 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:14:03.389194    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12ecd131-n7d7m"]
Sep 14 03:14:03.746482 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:14:03.746454    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bae37c3c-ace6-453c-a083-876a010df969" path="/var/lib/kubelet/pods/bae37c3c-ace6-453c-a083-876a010df969/volumes"
Sep 14 03:14:17.897797 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:14:17.897763    2579 scope.go:117] "RemoveContainer" containerID="afddc7a1185ea91d1cf23af2a1c105200f3bec824815a21591fcaf790b3ff546"
Sep 14 03:14:54.781785 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:14:54.781705    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"]
Sep 14 03:14:54.782375 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:14:54.781948    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w" podUID="265572f5-7a69-43a0-879d-62abe5b85d5f" containerName="istio-proxy" containerID="cri-o://32b7563568c8348e44e3bded31190b46fcb4fb5cb013a1a021a25fbeaba85322" gracePeriod=30
Sep 14 03:15:00.045038 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.044983    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:15:00.061197 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.061149    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-credential-socket\") pod \"265572f5-7a69-43a0-879d-62abe5b85d5f\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") "
Sep 14 03:15:00.061321 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.061206    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-envoy\") pod \"265572f5-7a69-43a0-879d-62abe5b85d5f\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") "
Sep 14 03:15:00.061321 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.061254    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/265572f5-7a69-43a0-879d-62abe5b85d5f-istiod-ca-cert\") pod \"265572f5-7a69-43a0-879d-62abe5b85d5f\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") "
Sep 14 03:15:00.061321 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.061306    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-podinfo\") pod \"265572f5-7a69-43a0-879d-62abe5b85d5f\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") "
Sep 14 03:15:00.061492 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.061339    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzmg9\" (UniqueName: \"kubernetes.io/projected/265572f5-7a69-43a0-879d-62abe5b85d5f-kube-api-access-zzmg9\") pod \"265572f5-7a69-43a0-879d-62abe5b85d5f\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") "
Sep 14 03:15:00.061492 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.061369    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-workload-socket\") pod \"265572f5-7a69-43a0-879d-62abe5b85d5f\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") "
Sep 14 03:15:00.061492 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.061409    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-workload-certs\") pod \"265572f5-7a69-43a0-879d-62abe5b85d5f\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") "
Sep 14 03:15:00.061492 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.061420    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "265572f5-7a69-43a0-879d-62abe5b85d5f" (UID: "265572f5-7a69-43a0-879d-62abe5b85d5f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:15:00.061492 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.061447    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-token\") pod \"265572f5-7a69-43a0-879d-62abe5b85d5f\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") "
Sep 14 03:15:00.061751 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.061506    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-data\") pod \"265572f5-7a69-43a0-879d-62abe5b85d5f\" (UID: \"265572f5-7a69-43a0-879d-62abe5b85d5f\") "
Sep 14 03:15:00.061751 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.061617    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "265572f5-7a69-43a0-879d-62abe5b85d5f" (UID: "265572f5-7a69-43a0-879d-62abe5b85d5f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:15:00.061852 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.061800    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/265572f5-7a69-43a0-879d-62abe5b85d5f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "265572f5-7a69-43a0-879d-62abe5b85d5f" (UID: "265572f5-7a69-43a0-879d-62abe5b85d5f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:15:00.061852 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.061828    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "265572f5-7a69-43a0-879d-62abe5b85d5f" (UID: "265572f5-7a69-43a0-879d-62abe5b85d5f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:15:00.061969 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.061930    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:15:00.062030 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.062007    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-data" (OuterVolumeSpecName: "istio-data") pod "265572f5-7a69-43a0-879d-62abe5b85d5f" (UID: "265572f5-7a69-43a0-879d-62abe5b85d5f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:15:00.062088 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.062032    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:15:00.062088 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.062051    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:15:00.062088 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.062066    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/265572f5-7a69-43a0-879d-62abe5b85d5f-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:15:00.063707 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.063676    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/265572f5-7a69-43a0-879d-62abe5b85d5f-kube-api-access-zzmg9" (OuterVolumeSpecName: "kube-api-access-zzmg9") pod "265572f5-7a69-43a0-879d-62abe5b85d5f" (UID: "265572f5-7a69-43a0-879d-62abe5b85d5f"). InnerVolumeSpecName "kube-api-access-zzmg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:15:00.063819 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.063724    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-token" (OuterVolumeSpecName: "istio-token") pod "265572f5-7a69-43a0-879d-62abe5b85d5f" (UID: "265572f5-7a69-43a0-879d-62abe5b85d5f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:15:00.063929 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.063911    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "265572f5-7a69-43a0-879d-62abe5b85d5f" (UID: "265572f5-7a69-43a0-879d-62abe5b85d5f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:15:00.064122 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.064104    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "265572f5-7a69-43a0-879d-62abe5b85d5f" (UID: "265572f5-7a69-43a0-879d-62abe5b85d5f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:15:00.161524 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.161494    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-177593-29822595-nt4d8"]
Sep 14 03:15:00.161893 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.161870    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bae37c3c-ace6-453c-a083-876a010df969" containerName="revoke-keys"
Sep 14 03:15:00.161938 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.161895    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bae37c3c-ace6-453c-a083-876a010df969" containerName="revoke-keys"
Sep 14 03:15:00.161938 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.161904    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="265572f5-7a69-43a0-879d-62abe5b85d5f" containerName="istio-proxy"
Sep 14 03:15:00.161938 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.161909    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="265572f5-7a69-43a0-879d-62abe5b85d5f" containerName="istio-proxy"
Sep 14 03:15:00.161938 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.161920    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="265572f5-7a69-43a0-879d-62abe5b85d5f" containerName="istio-proxy"
Sep 14 03:15:00.161938 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.161926    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="265572f5-7a69-43a0-879d-62abe5b85d5f" containerName="istio-proxy"
Sep 14 03:15:00.162092 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.161997    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bae37c3c-ace6-453c-a083-876a010df969" containerName="revoke-keys"
Sep 14 03:15:00.162092 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.162006    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="265572f5-7a69-43a0-879d-62abe5b85d5f" containerName="istio-proxy"
Sep 14 03:15:00.162092 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.162013    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="265572f5-7a69-43a0-879d-62abe5b85d5f" containerName="istio-proxy"
Sep 14 03:15:00.162816 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.162798    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:15:00.162861 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.162820    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:15:00.162861 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.162828    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:15:00.162861 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.162836    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/265572f5-7a69-43a0-879d-62abe5b85d5f-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:15:00.162861 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.162844    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzmg9\" (UniqueName: \"kubernetes.io/projected/265572f5-7a69-43a0-879d-62abe5b85d5f-kube-api-access-zzmg9\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:15:00.165167 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.165149    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-177593-29822595-nt4d8"
Sep 14 03:15:00.169440 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.169423    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:15:00.175523 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.175503    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-177593-29822595-nt4d8"
Sep 14 03:15:00.176579 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.176561    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ce5b9b-29822595-h6kjt"]
Sep 14 03:15:00.179818 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.179800    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-3a752f-29822595-dlmpm"]
Sep 14 03:15:00.179923 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.179906    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ce5b9b-29822595-h6kjt"
Sep 14 03:15:00.183791 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.183771    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-3a752f-29822595-dlmpm"
Sep 14 03:15:00.189483 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.189467    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ce5b9b-29822595-h6kjt"
Sep 14 03:15:00.197902 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.197871    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-3a752f-29822595-dlmpm"
Sep 14 03:15:00.199914 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.199833    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-177593-29822595-nt4d8"]
Sep 14 03:15:00.206723 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.206688    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ce5b9b-29822595-h6kjt"]
Sep 14 03:15:00.235120 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.234395    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822595-8pmv6"]
Sep 14 03:15:00.260061 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.260033    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-3a752f-29822595-dlmpm"]
Sep 14 03:15:00.260449 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.260434    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822595-8pmv6"
Sep 14 03:15:00.267333 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.267039    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822595-8pmv6"]
Sep 14 03:15:00.286843 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.286666    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822595-8pmv6"
Sep 14 03:15:00.485580 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.485542    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-177593-29822595-nt4d8"]
Sep 14 03:15:00.486794 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:15:00.486755    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod065da8a6_6109_4130_9955_0334ca40c0a6.slice/crio-a4d6b98f0d95a8a19a3e512d87e893d8cb5666c62c2a2ecd729811abcad48362 WatchSource:0}: Error finding container a4d6b98f0d95a8a19a3e512d87e893d8cb5666c62c2a2ecd729811abcad48362: Status 404 returned error can't find the container with id a4d6b98f0d95a8a19a3e512d87e893d8cb5666c62c2a2ecd729811abcad48362
Sep 14 03:15:00.507571 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.507545    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ce5b9b-29822595-h6kjt"]
Sep 14 03:15:00.507969 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:15:00.507943    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82a1f589_deb7_410d_8627_2bb120c63c23.slice/crio-125a511b4f9e126ac5c3425586fe6e8bf98966c83e8099ed9db3f10b9704702f WatchSource:0}: Error finding container 125a511b4f9e126ac5c3425586fe6e8bf98966c83e8099ed9db3f10b9704702f: Status 404 returned error can't find the container with id 125a511b4f9e126ac5c3425586fe6e8bf98966c83e8099ed9db3f10b9704702f
Sep 14 03:15:00.517793 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.517772    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-3a752f-29822595-dlmpm"]
Sep 14 03:15:00.521542 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:15:00.521518    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd170a489_d7ec_40e8_b6fc_426a564ced36.slice/crio-d2b5935faccb66e7659ab86c01d91b12b1240e8c0abbd3262d83d26e102833f0 WatchSource:0}: Error finding container d2b5935faccb66e7659ab86c01d91b12b1240e8c0abbd3262d83d26e102833f0: Status 404 returned error can't find the container with id d2b5935faccb66e7659ab86c01d91b12b1240e8c0abbd3262d83d26e102833f0
Sep 14 03:15:00.564130 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.564073    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-3a752f-29822595-dlmpm" event={"ID":"d170a489-d7ec-40e8-b6fc-426a564ced36","Type":"ContainerStarted","Data":"d2b5935faccb66e7659ab86c01d91b12b1240e8c0abbd3262d83d26e102833f0"}
Sep 14 03:15:00.565636 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.565612    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-177593-29822595-nt4d8" event={"ID":"065da8a6-6109-4130-9955-0334ca40c0a6","Type":"ContainerStarted","Data":"a4d6b98f0d95a8a19a3e512d87e893d8cb5666c62c2a2ecd729811abcad48362"}
Sep 14 03:15:00.566929 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.566908    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ce5b9b-29822595-h6kjt" event={"ID":"82a1f589-deb7-410d-8627-2bb120c63c23","Type":"ContainerStarted","Data":"125a511b4f9e126ac5c3425586fe6e8bf98966c83e8099ed9db3f10b9704702f"}
Sep 14 03:15:00.568678 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.568657    2579 generic.go:356] "Generic (PLEG): container finished" podID="265572f5-7a69-43a0-879d-62abe5b85d5f" containerID="32b7563568c8348e44e3bded31190b46fcb4fb5cb013a1a021a25fbeaba85322" exitCode=0
Sep 14 03:15:00.568775 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.568706    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w" event={"ID":"265572f5-7a69-43a0-879d-62abe5b85d5f","Type":"ContainerDied","Data":"32b7563568c8348e44e3bded31190b46fcb4fb5cb013a1a021a25fbeaba85322"}
Sep 14 03:15:00.568775 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.568719    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"
Sep 14 03:15:00.568775 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.568735    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w" event={"ID":"265572f5-7a69-43a0-879d-62abe5b85d5f","Type":"ContainerDied","Data":"21d7343fb7ebd3b514cd565326bd44e86f515584970aea441cf2fa5b9c1b5de0"}
Sep 14 03:15:00.568775 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.568755    2579 scope.go:117] "RemoveContainer" containerID="32b7563568c8348e44e3bded31190b46fcb4fb5cb013a1a021a25fbeaba85322"
Sep 14 03:15:00.577532 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.577518    2579 scope.go:117] "RemoveContainer" containerID="eb28d3ef1bd9e7427320045927585b3023de391570cc2853f6f85c4bcd17b39a"
Sep 14 03:15:00.585718 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.585676    2579 scope.go:117] "RemoveContainer" containerID="32b7563568c8348e44e3bded31190b46fcb4fb5cb013a1a021a25fbeaba85322"
Sep 14 03:15:00.586327 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:15:00.586301    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32b7563568c8348e44e3bded31190b46fcb4fb5cb013a1a021a25fbeaba85322\": container with ID starting with 32b7563568c8348e44e3bded31190b46fcb4fb5cb013a1a021a25fbeaba85322 not found: ID does not exist" containerID="32b7563568c8348e44e3bded31190b46fcb4fb5cb013a1a021a25fbeaba85322"
Sep 14 03:15:00.586408 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.586332    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32b7563568c8348e44e3bded31190b46fcb4fb5cb013a1a021a25fbeaba85322"} err="failed to get container status \"32b7563568c8348e44e3bded31190b46fcb4fb5cb013a1a021a25fbeaba85322\": rpc error: code = NotFound desc = could not find container \"32b7563568c8348e44e3bded31190b46fcb4fb5cb013a1a021a25fbeaba85322\": container with ID starting with 32b7563568c8348e44e3bded31190b46fcb4fb5cb013a1a021a25fbeaba85322 not found: ID does not exist"
Sep 14 03:15:00.586408 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.586347    2579 scope.go:117] "RemoveContainer" containerID="eb28d3ef1bd9e7427320045927585b3023de391570cc2853f6f85c4bcd17b39a"
Sep 14 03:15:00.586666 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:15:00.586646    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb28d3ef1bd9e7427320045927585b3023de391570cc2853f6f85c4bcd17b39a\": container with ID starting with eb28d3ef1bd9e7427320045927585b3023de391570cc2853f6f85c4bcd17b39a not found: ID does not exist" containerID="eb28d3ef1bd9e7427320045927585b3023de391570cc2853f6f85c4bcd17b39a"
Sep 14 03:15:00.586725 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.586676    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb28d3ef1bd9e7427320045927585b3023de391570cc2853f6f85c4bcd17b39a"} err="failed to get container status \"eb28d3ef1bd9e7427320045927585b3023de391570cc2853f6f85c4bcd17b39a\": rpc error: code = NotFound desc = could not find container \"eb28d3ef1bd9e7427320045927585b3023de391570cc2853f6f85c4bcd17b39a\": container with ID starting with eb28d3ef1bd9e7427320045927585b3023de391570cc2853f6f85c4bcd17b39a not found: ID does not exist"
Sep 14 03:15:00.588317 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.588301    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822595-8pmv6"]
Sep 14 03:15:00.589635 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:15:00.589613    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9651dd66_4bad_4bb3_a8c3_d6d3bc5d540d.slice/crio-d04ce19e0bb3cc2ea43e28c8da252c1ef4c486937fcf73f2b82f85755ee2ed3e WatchSource:0}: Error finding container d04ce19e0bb3cc2ea43e28c8da252c1ef4c486937fcf73f2b82f85755ee2ed3e: Status 404 returned error can't find the container with id d04ce19e0bb3cc2ea43e28c8da252c1ef4c486937fcf73f2b82f85755ee2ed3e
Sep 14 03:15:00.595703 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.595676    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"]
Sep 14 03:15:00.600523 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:00.600500    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-12ecd131-openshift-default-74fc8b59cb-ns82w"]
Sep 14 03:15:01.575989 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:01.575955    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-3a752f-29822595-dlmpm" event={"ID":"d170a489-d7ec-40e8-b6fc-426a564ced36","Type":"ContainerStarted","Data":"4bb5c38bed0f5d91a4e350f2cf9b3845dbf83786f2f1d6e10ad846934a040025"}
Sep 14 03:15:01.577632 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:01.577606    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822595-8pmv6" event={"ID":"9651dd66-4bad-4bb3-a8c3-d6d3bc5d540d","Type":"ContainerStarted","Data":"8ad2ea071d0f8fb6306158737f42892ca26c31819c672c31ada6e43f7f2474cb"}
Sep 14 03:15:01.577784 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:01.577651    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822595-8pmv6" event={"ID":"9651dd66-4bad-4bb3-a8c3-d6d3bc5d540d","Type":"ContainerStarted","Data":"d04ce19e0bb3cc2ea43e28c8da252c1ef4c486937fcf73f2b82f85755ee2ed3e"}
Sep 14 03:15:01.579141 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:01.579117    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-177593-29822595-nt4d8" event={"ID":"065da8a6-6109-4130-9955-0334ca40c0a6","Type":"ContainerStarted","Data":"0867d47a0a7ad5098202c0f80f2c477c40bf5e3bf567da91e56f01d16566f135"}
Sep 14 03:15:01.580701 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:01.580679    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ce5b9b-29822595-h6kjt" event={"ID":"82a1f589-deb7-410d-8627-2bb120c63c23","Type":"ContainerStarted","Data":"e48ffc00d5ec12eea834dc8515313f9b96d67cf6ecf903641fddc7eb97c31d06"}
Sep 14 03:15:01.597642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:01.597592    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-3a752f-29822595-dlmpm" podStartSLOduration=1.5975743740000001 podStartE2EDuration="1.597574374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:01.595203551 +0000 UTC m=+1364.381265471" watchObservedRunningTime="2026-09-14 03:15:01.597574374 +0000 UTC m=+1364.383636288"
Sep 14 03:15:01.614653 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:01.614617    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-177593-29822595-nt4d8" podStartSLOduration=1.614604986 podStartE2EDuration="1.614604986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:01.612542801 +0000 UTC m=+1364.398604716" watchObservedRunningTime="2026-09-14 03:15:01.614604986 +0000 UTC m=+1364.400666898"
Sep 14 03:15:01.629793 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:01.629756    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ce5b9b-29822595-h6kjt" podStartSLOduration=1.629744372 podStartE2EDuration="1.629744372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:01.628045732 +0000 UTC m=+1364.414107646" watchObservedRunningTime="2026-09-14 03:15:01.629744372 +0000 UTC m=+1364.415806284"
Sep 14 03:15:01.643632 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:01.643598    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822595-8pmv6" podStartSLOduration=1.643588574 podStartE2EDuration="1.643588574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:01.643457321 +0000 UTC m=+1364.429519238" watchObservedRunningTime="2026-09-14 03:15:01.643588574 +0000 UTC m=+1364.429650487"
Sep 14 03:15:01.744103 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:01.744080    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="265572f5-7a69-43a0-879d-62abe5b85d5f" path="/var/lib/kubelet/pods/265572f5-7a69-43a0-879d-62abe5b85d5f/volumes"
Sep 14 03:15:02.589287 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:02.589254    2579 generic.go:356] "Generic (PLEG): container finished" podID="065da8a6-6109-4130-9955-0334ca40c0a6" containerID="0867d47a0a7ad5098202c0f80f2c477c40bf5e3bf567da91e56f01d16566f135" exitCode=0
Sep 14 03:15:02.589710 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:02.589333    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-177593-29822595-nt4d8" event={"ID":"065da8a6-6109-4130-9955-0334ca40c0a6","Type":"ContainerDied","Data":"0867d47a0a7ad5098202c0f80f2c477c40bf5e3bf567da91e56f01d16566f135"}
Sep 14 03:15:02.591046 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:02.591024    2579 generic.go:356] "Generic (PLEG): container finished" podID="82a1f589-deb7-410d-8627-2bb120c63c23" containerID="e48ffc00d5ec12eea834dc8515313f9b96d67cf6ecf903641fddc7eb97c31d06" exitCode=0
Sep 14 03:15:02.591199 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:02.591105    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ce5b9b-29822595-h6kjt" event={"ID":"82a1f589-deb7-410d-8627-2bb120c63c23","Type":"ContainerDied","Data":"e48ffc00d5ec12eea834dc8515313f9b96d67cf6ecf903641fddc7eb97c31d06"}
Sep 14 03:15:02.592859 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:02.592839    2579 generic.go:356] "Generic (PLEG): container finished" podID="d170a489-d7ec-40e8-b6fc-426a564ced36" containerID="4bb5c38bed0f5d91a4e350f2cf9b3845dbf83786f2f1d6e10ad846934a040025" exitCode=0
Sep 14 03:15:02.592974 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:02.592906    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-3a752f-29822595-dlmpm" event={"ID":"d170a489-d7ec-40e8-b6fc-426a564ced36","Type":"ContainerDied","Data":"4bb5c38bed0f5d91a4e350f2cf9b3845dbf83786f2f1d6e10ad846934a040025"}
Sep 14 03:15:02.594693 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:02.594671    2579 generic.go:356] "Generic (PLEG): container finished" podID="9651dd66-4bad-4bb3-a8c3-d6d3bc5d540d" containerID="8ad2ea071d0f8fb6306158737f42892ca26c31819c672c31ada6e43f7f2474cb" exitCode=0
Sep 14 03:15:02.594817 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:02.594697    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822595-8pmv6" event={"ID":"9651dd66-4bad-4bb3-a8c3-d6d3bc5d540d","Type":"ContainerDied","Data":"8ad2ea071d0f8fb6306158737f42892ca26c31819c672c31ada6e43f7f2474cb"}
Sep 14 03:15:03.751247 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:03.751220    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ce5b9b-29822595-h6kjt"
Sep 14 03:15:03.799155 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:03.799132    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-177593-29822595-nt4d8"
Sep 14 03:15:03.840460 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:03.840439    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822595-8pmv6"
Sep 14 03:15:03.845120 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:03.845088    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-3a752f-29822595-dlmpm"
Sep 14 03:15:04.606309 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:04.606278    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-3a752f-29822595-dlmpm"
Sep 14 03:15:04.606495 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:04.606280    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-3a752f-29822595-dlmpm" event={"ID":"d170a489-d7ec-40e8-b6fc-426a564ced36","Type":"ContainerDied","Data":"d2b5935faccb66e7659ab86c01d91b12b1240e8c0abbd3262d83d26e102833f0"}
Sep 14 03:15:04.606495 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:04.606400    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2b5935faccb66e7659ab86c01d91b12b1240e8c0abbd3262d83d26e102833f0"
Sep 14 03:15:04.607718 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:04.607686    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822595-8pmv6"
Sep 14 03:15:04.607718 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:04.607709    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822595-8pmv6" event={"ID":"9651dd66-4bad-4bb3-a8c3-d6d3bc5d540d","Type":"ContainerDied","Data":"d04ce19e0bb3cc2ea43e28c8da252c1ef4c486937fcf73f2b82f85755ee2ed3e"}
Sep 14 03:15:04.607916 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:04.607747    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d04ce19e0bb3cc2ea43e28c8da252c1ef4c486937fcf73f2b82f85755ee2ed3e"
Sep 14 03:15:04.609265 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:04.609249    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-177593-29822595-nt4d8"
Sep 14 03:15:04.609352 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:04.609251    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-177593-29822595-nt4d8" event={"ID":"065da8a6-6109-4130-9955-0334ca40c0a6","Type":"ContainerDied","Data":"a4d6b98f0d95a8a19a3e512d87e893d8cb5666c62c2a2ecd729811abcad48362"}
Sep 14 03:15:04.609352 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:04.609302    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4d6b98f0d95a8a19a3e512d87e893d8cb5666c62c2a2ecd729811abcad48362"
Sep 14 03:15:04.610880 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:04.610858    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ce5b9b-29822595-h6kjt" event={"ID":"82a1f589-deb7-410d-8627-2bb120c63c23","Type":"ContainerDied","Data":"125a511b4f9e126ac5c3425586fe6e8bf98966c83e8099ed9db3f10b9704702f"}
Sep 14 03:15:04.610978 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:04.610900    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="125a511b4f9e126ac5c3425586fe6e8bf98966c83e8099ed9db3f10b9704702f"
Sep 14 03:15:04.610978 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:04.610922    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ce5b9b-29822595-h6kjt"
Sep 14 03:15:49.940438 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:49.940407    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"]
Sep 14 03:15:49.941037 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:49.940988    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9651dd66-4bad-4bb3-a8c3-d6d3bc5d540d" containerName="cleanup"
Sep 14 03:15:49.941037 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:49.941007    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9651dd66-4bad-4bb3-a8c3-d6d3bc5d540d" containerName="cleanup"
Sep 14 03:15:49.941037 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:49.941023    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d170a489-d7ec-40e8-b6fc-426a564ced36" containerName="cleanup"
Sep 14 03:15:49.941037 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:49.941033    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d170a489-d7ec-40e8-b6fc-426a564ced36" containerName="cleanup"
Sep 14 03:15:49.941287 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:49.941052    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82a1f589-deb7-410d-8627-2bb120c63c23" containerName="cleanup"
Sep 14 03:15:49.941287 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:49.941060    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="82a1f589-deb7-410d-8627-2bb120c63c23" containerName="cleanup"
Sep 14 03:15:49.941287 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:49.941088    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="065da8a6-6109-4130-9955-0334ca40c0a6" containerName="cleanup"
Sep 14 03:15:49.941287 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:49.941097    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="065da8a6-6109-4130-9955-0334ca40c0a6" containerName="cleanup"
Sep 14 03:15:49.941287 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:49.941213    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="065da8a6-6109-4130-9955-0334ca40c0a6" containerName="cleanup"
Sep 14 03:15:49.941287 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:49.941223    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d170a489-d7ec-40e8-b6fc-426a564ced36" containerName="cleanup"
Sep 14 03:15:49.941287 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:49.941230    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9651dd66-4bad-4bb3-a8c3-d6d3bc5d540d" containerName="cleanup"
Sep 14 03:15:49.941287 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:49.941240    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="82a1f589-deb7-410d-8627-2bb120c63c23" containerName="cleanup"
Sep 14 03:15:49.944587 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:49.944566    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:49.949541 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:49.949516    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-3a33c1-openshift-default-dockercfg-fj494\""
Sep 14 03:15:49.956817 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:49.956793    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"]
Sep 14 03:15:50.016928 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.016873    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-data\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.017058 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.016939    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5de2715c-fdc1-4537-9bae-530e918e5ba5-istiod-ca-cert\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.017058 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.016993    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-workload-certs\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.017058 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.017051    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-envoy\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.017162 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.017084    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-podinfo\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.017162 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.017112    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-credential-socket\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.017162 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.017146    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-workload-socket\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.017299 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.017211    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-token\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.017299 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.017241    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzfwh\" (UniqueName: \"kubernetes.io/projected/5de2715c-fdc1-4537-9bae-530e918e5ba5-kube-api-access-wzfwh\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.118119 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.118087    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-token\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.118334 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.118128    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzfwh\" (UniqueName: \"kubernetes.io/projected/5de2715c-fdc1-4537-9bae-530e918e5ba5-kube-api-access-wzfwh\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.118334 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.118167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-data\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.118334 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.118229    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5de2715c-fdc1-4537-9bae-530e918e5ba5-istiod-ca-cert\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.118334 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.118249    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-workload-certs\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.118334 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.118303    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-envoy\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.118615 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.118341    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-podinfo\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.118615 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.118489    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-credential-socket\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.118615 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.118545    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-workload-socket\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.118773 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.118672    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-data\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.118773 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.118754    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-workload-certs\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.118977 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.118953    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-credential-socket\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.119041 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.118970    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-workload-socket\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.119041 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.119023    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5de2715c-fdc1-4537-9bae-530e918e5ba5-istiod-ca-cert\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.121329 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.121303    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-envoy\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.121489 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.121475    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-podinfo\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.126757 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.126734    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-token\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.126880 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.126861    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzfwh\" (UniqueName: \"kubernetes.io/projected/5de2715c-fdc1-4537-9bae-530e918e5ba5-kube-api-access-wzfwh\") pod \"e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") " pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.257615 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.257581    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:50.544900 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.544871    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"]
Sep 14 03:15:50.547738 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:15:50.547694    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5de2715c_fdc1_4537_9bae_530e918e5ba5.slice/crio-50b07c4b2685585c49ce34a5449450b4f72c263c3dd9c14772e1b6e4ba81299f WatchSource:0}: Error finding container 50b07c4b2685585c49ce34a5449450b4f72c263c3dd9c14772e1b6e4ba81299f: Status 404 returned error can't find the container with id 50b07c4b2685585c49ce34a5449450b4f72c263c3dd9c14772e1b6e4ba81299f
Sep 14 03:15:50.550313 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.550271    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:15:50.550392 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.550371    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:15:50.550432 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.550418    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:15:50.868246 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.868130    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww" event={"ID":"5de2715c-fdc1-4537-9bae-530e918e5ba5","Type":"ContainerStarted","Data":"140b7181ddb27978ca0765dcac6f2d8bf83143db42d7842270e4ee5e2c13a26e"}
Sep 14 03:15:50.868246 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.868217    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww" event={"ID":"5de2715c-fdc1-4537-9bae-530e918e5ba5","Type":"ContainerStarted","Data":"50b07c4b2685585c49ce34a5449450b4f72c263c3dd9c14772e1b6e4ba81299f"}
Sep 14 03:15:50.893443 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:50.893399    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww" podStartSLOduration=1.893386029 podStartE2EDuration="1.893386029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:50.890458908 +0000 UTC m=+1413.676520821" watchObservedRunningTime="2026-09-14 03:15:50.893386029 +0000 UTC m=+1413.679447942"
Sep 14 03:15:51.258163 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:51.258130    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:51.263306 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:51.263278    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:51.873766 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:51.873728    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:51.874791 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:51.874771    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:15:58.657507 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:58.657472    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"]
Sep 14 03:15:58.661215 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:58.661191    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"
Sep 14 03:15:58.663719 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:58.663695    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-3a33c1-dockercfg-q47wv\""
Sep 14 03:15:58.663719 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:58.663715    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-3a33c1\""
Sep 14 03:15:58.673737 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:58.673714    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"]
Sep 14 03:15:58.703642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:58.703615    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04c70921-127d-464b-9cad-49bef8598ffd-plugins-config-volume\") pod \"payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg\" (UID: \"04c70921-127d-464b-9cad-49bef8598ffd\") " pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"
Sep 14 03:15:58.703797 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:58.703675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdf7w\" (UniqueName: \"kubernetes.io/projected/04c70921-127d-464b-9cad-49bef8598ffd-kube-api-access-xdf7w\") pod \"payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg\" (UID: \"04c70921-127d-464b-9cad-49bef8598ffd\") " pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"
Sep 14 03:15:58.805129 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:58.805082    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04c70921-127d-464b-9cad-49bef8598ffd-plugins-config-volume\") pod \"payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg\" (UID: \"04c70921-127d-464b-9cad-49bef8598ffd\") " pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"
Sep 14 03:15:58.805349 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:58.805192    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdf7w\" (UniqueName: \"kubernetes.io/projected/04c70921-127d-464b-9cad-49bef8598ffd-kube-api-access-xdf7w\") pod \"payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg\" (UID: \"04c70921-127d-464b-9cad-49bef8598ffd\") " pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"
Sep 14 03:15:58.806150 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:58.806126    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04c70921-127d-464b-9cad-49bef8598ffd-plugins-config-volume\") pod \"payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg\" (UID: \"04c70921-127d-464b-9cad-49bef8598ffd\") " pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"
Sep 14 03:15:58.814129 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:58.814098    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdf7w\" (UniqueName: \"kubernetes.io/projected/04c70921-127d-464b-9cad-49bef8598ffd-kube-api-access-xdf7w\") pod \"payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg\" (UID: \"04c70921-127d-464b-9cad-49bef8598ffd\") " pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"
Sep 14 03:15:58.974734 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:58.974651    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"
Sep 14 03:15:59.253320 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:59.253292    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"]
Sep 14 03:15:59.254388 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:15:59.254365    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04c70921_127d_464b_9cad_49bef8598ffd.slice/crio-ae2059d69c98e23febfa64c0c1fc6147dc561e6dacb9524c7396b4dce31336d4 WatchSource:0}: Error finding container ae2059d69c98e23febfa64c0c1fc6147dc561e6dacb9524c7396b4dce31336d4: Status 404 returned error can't find the container with id ae2059d69c98e23febfa64c0c1fc6147dc561e6dacb9524c7396b4dce31336d4
Sep 14 03:15:59.924602 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:59.924571    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg" event={"ID":"04c70921-127d-464b-9cad-49bef8598ffd","Type":"ContainerStarted","Data":"d268d0b686d1913211687353acf2620e91f901e5eb78e073d5b121421787b9ce"}
Sep 14 03:15:59.925012 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:59.924628    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg" event={"ID":"04c70921-127d-464b-9cad-49bef8598ffd","Type":"ContainerStarted","Data":"ae2059d69c98e23febfa64c0c1fc6147dc561e6dacb9524c7396b4dce31336d4"}
Sep 14 03:15:59.925012 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:59.924736    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"
Sep 14 03:15:59.944054 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:15:59.944005    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg" podStartSLOduration=1.943989747 podStartE2EDuration="1.943989747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:15:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:15:59.941901538 +0000 UTC m=+1422.727963453" watchObservedRunningTime="2026-09-14 03:15:59.943989747 +0000 UTC m=+1422.730051686"
Sep 14 03:16:10.933589 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:10.933515    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"
Sep 14 03:16:26.352070 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:26.352031    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd"]
Sep 14 03:16:26.355984 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:26.355965    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd"
Sep 14 03:16:26.358570 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:26.358542    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:16:26.365462 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:26.365426    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd"]
Sep 14 03:16:26.476418 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:26.476382    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a570fd4-4e1a-4ce2-a36b-1d8313991709-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd\" (UID: \"1a570fd4-4e1a-4ce2-a36b-1d8313991709\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd"
Sep 14 03:16:26.577698 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:26.577667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a570fd4-4e1a-4ce2-a36b-1d8313991709-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd\" (UID: \"1a570fd4-4e1a-4ce2-a36b-1d8313991709\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd"
Sep 14 03:16:26.578506 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:26.578483    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a570fd4-4e1a-4ce2-a36b-1d8313991709-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd\" (UID: \"1a570fd4-4e1a-4ce2-a36b-1d8313991709\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd"
Sep 14 03:16:26.668982 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:26.668925    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd"
Sep 14 03:16:26.962362 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:26.962333    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd"]
Sep 14 03:16:26.963862 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:16:26.963835    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a570fd4_4e1a_4ce2_a36b_1d8313991709.slice/crio-4ca7ace3bdd15a745adce66711850b7d17538f66ee9ab182d6f8b5946f6413d4 WatchSource:0}: Error finding container 4ca7ace3bdd15a745adce66711850b7d17538f66ee9ab182d6f8b5946f6413d4: Status 404 returned error can't find the container with id 4ca7ace3bdd15a745adce66711850b7d17538f66ee9ab182d6f8b5946f6413d4
Sep 14 03:16:27.080044 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:27.080010    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd" event={"ID":"1a570fd4-4e1a-4ce2-a36b-1d8313991709","Type":"ContainerStarted","Data":"4ca7ace3bdd15a745adce66711850b7d17538f66ee9ab182d6f8b5946f6413d4"}
Sep 14 03:16:28.087285 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:28.087247    2579 generic.go:356] "Generic (PLEG): container finished" podID="1a570fd4-4e1a-4ce2-a36b-1d8313991709" containerID="c82d2390e576bddd8ba764d21a0882e13b80788ccb5bfe35c06196680bb9d17a" exitCode=0
Sep 14 03:16:28.087668 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:28.087292    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd" event={"ID":"1a570fd4-4e1a-4ce2-a36b-1d8313991709","Type":"ContainerDied","Data":"c82d2390e576bddd8ba764d21a0882e13b80788ccb5bfe35c06196680bb9d17a"}
Sep 14 03:16:29.243731 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:29.243706    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd"
Sep 14 03:16:29.303675 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:29.303634    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a570fd4-4e1a-4ce2-a36b-1d8313991709-maas-api-service-ca\") pod \"1a570fd4-4e1a-4ce2-a36b-1d8313991709\" (UID: \"1a570fd4-4e1a-4ce2-a36b-1d8313991709\") "
Sep 14 03:16:29.304020 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:29.303997    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a570fd4-4e1a-4ce2-a36b-1d8313991709-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1a570fd4-4e1a-4ce2-a36b-1d8313991709" (UID: "1a570fd4-4e1a-4ce2-a36b-1d8313991709"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:16:29.405246 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:29.405139    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a570fd4-4e1a-4ce2-a36b-1d8313991709-maas-api-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:16:30.102639 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:30.102544    2579 generic.go:356] "Generic (PLEG): container finished" podID="5de2715c-fdc1-4537-9bae-530e918e5ba5" containerID="140b7181ddb27978ca0765dcac6f2d8bf83143db42d7842270e4ee5e2c13a26e" exitCode=0
Sep 14 03:16:30.102639 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:30.102622    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww" event={"ID":"5de2715c-fdc1-4537-9bae-530e918e5ba5","Type":"ContainerDied","Data":"140b7181ddb27978ca0765dcac6f2d8bf83143db42d7842270e4ee5e2c13a26e"}
Sep 14 03:16:30.103109 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:30.103092    2579 scope.go:117] "RemoveContainer" containerID="140b7181ddb27978ca0765dcac6f2d8bf83143db42d7842270e4ee5e2c13a26e"
Sep 14 03:16:30.104265 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:30.104241    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd"
Sep 14 03:16:30.104381 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:30.104265    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd" event={"ID":"1a570fd4-4e1a-4ce2-a36b-1d8313991709","Type":"ContainerDied","Data":"4ca7ace3bdd15a745adce66711850b7d17538f66ee9ab182d6f8b5946f6413d4"}
Sep 14 03:16:30.104381 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:30.104305    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ca7ace3bdd15a745adce66711850b7d17538f66ee9ab182d6f8b5946f6413d4"
Sep 14 03:16:30.104592 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:30.104566    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:16:30.104667 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:30.104648    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:16:30.104733 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:30.104702    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:16:31.111942 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:31.111912    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww" event={"ID":"5de2715c-fdc1-4537-9bae-530e918e5ba5","Type":"ContainerStarted","Data":"9430259db73b2fb81bad80b1564065df9275e42652e17680dce89aabf7b9cea5"}
Sep 14 03:16:31.258836 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:31.258809    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:16:31.259055 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:31.259034    2579 patch_prober.go:28] interesting pod/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 14 03:16:31.259139 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:31.259087    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww" podUID="5de2715c-fdc1-4537-9bae-530e918e5ba5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 14 03:16:32.258636 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:32.258604    2579 patch_prober.go:28] interesting pod/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 14 03:16:32.259004 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:32.258664    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww" podUID="5de2715c-fdc1-4537-9bae-530e918e5ba5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 14 03:16:33.258352 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:33.258318    2579 patch_prober.go:28] interesting pod/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 14 03:16:33.258528 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:33.258388    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww" podUID="5de2715c-fdc1-4537-9bae-530e918e5ba5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 14 03:16:34.225991 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.225954    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"]
Sep 14 03:16:34.226650 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.226628    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a570fd4-4e1a-4ce2-a36b-1d8313991709" containerName="revoke-keys"
Sep 14 03:16:34.226732 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.226652    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a570fd4-4e1a-4ce2-a36b-1d8313991709" containerName="revoke-keys"
Sep 14 03:16:34.226791 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.226774    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a570fd4-4e1a-4ce2-a36b-1d8313991709" containerName="revoke-keys"
Sep 14 03:16:34.229509 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.229489    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"
Sep 14 03:16:34.233893 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.233866    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-97da78\""
Sep 14 03:16:34.233999 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.233967    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-97da78-dockercfg-2x8xw\""
Sep 14 03:16:34.243775 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.243746    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"]
Sep 14 03:16:34.253399 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.253378    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/493ba152-460b-4780-ae56-e56bbc62ad40-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95\" (UID: \"493ba152-460b-4780-ae56-e56bbc62ad40\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"
Sep 14 03:16:34.253545 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.253522    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shw44\" (UniqueName: \"kubernetes.io/projected/493ba152-460b-4780-ae56-e56bbc62ad40-kube-api-access-shw44\") pod \"payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95\" (UID: \"493ba152-460b-4780-ae56-e56bbc62ad40\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"
Sep 14 03:16:34.258454 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.258433    2579 patch_prober.go:28] interesting pod/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 14 03:16:34.258553 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.258492    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww" podUID="5de2715c-fdc1-4537-9bae-530e918e5ba5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 14 03:16:34.354970 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.354933    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shw44\" (UniqueName: \"kubernetes.io/projected/493ba152-460b-4780-ae56-e56bbc62ad40-kube-api-access-shw44\") pod \"payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95\" (UID: \"493ba152-460b-4780-ae56-e56bbc62ad40\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"
Sep 14 03:16:34.355138 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.355008    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/493ba152-460b-4780-ae56-e56bbc62ad40-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95\" (UID: \"493ba152-460b-4780-ae56-e56bbc62ad40\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"
Sep 14 03:16:34.355737 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.355719    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/493ba152-460b-4780-ae56-e56bbc62ad40-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95\" (UID: \"493ba152-460b-4780-ae56-e56bbc62ad40\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"
Sep 14 03:16:34.364906 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.364884    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shw44\" (UniqueName: \"kubernetes.io/projected/493ba152-460b-4780-ae56-e56bbc62ad40-kube-api-access-shw44\") pod \"payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95\" (UID: \"493ba152-460b-4780-ae56-e56bbc62ad40\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"
Sep 14 03:16:34.539949 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.539915    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"
Sep 14 03:16:34.822275 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:34.822112    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"]
Sep 14 03:16:34.824671 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:16:34.824640    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod493ba152_460b_4780_ae56_e56bbc62ad40.slice/crio-dfb5263be6f26efae62a2fda54484a06ae43e8b6b2e06d002d77421046d7cba4 WatchSource:0}: Error finding container dfb5263be6f26efae62a2fda54484a06ae43e8b6b2e06d002d77421046d7cba4: Status 404 returned error can't find the container with id dfb5263be6f26efae62a2fda54484a06ae43e8b6b2e06d002d77421046d7cba4
Sep 14 03:16:35.140830 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:35.140741    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95" event={"ID":"493ba152-460b-4780-ae56-e56bbc62ad40","Type":"ContainerStarted","Data":"1cec149c0502a82161ac6fc8018bb98b48485e64dbef812223db5ab0a94b27e1"}
Sep 14 03:16:35.140830 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:35.140804    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95" event={"ID":"493ba152-460b-4780-ae56-e56bbc62ad40","Type":"ContainerStarted","Data":"dfb5263be6f26efae62a2fda54484a06ae43e8b6b2e06d002d77421046d7cba4"}
Sep 14 03:16:35.141017 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:35.140836    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"
Sep 14 03:16:35.158678 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:35.158633    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95" podStartSLOduration=1.158617625 podStartE2EDuration="1.158617625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:16:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:16:35.157341882 +0000 UTC m=+1457.943403796" watchObservedRunningTime="2026-09-14 03:16:35.158617625 +0000 UTC m=+1457.944679538"
Sep 14 03:16:35.258138 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:35.258103    2579 patch_prober.go:28] interesting pod/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 14 03:16:35.258535 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:35.258156    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww" podUID="5de2715c-fdc1-4537-9bae-530e918e5ba5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 14 03:16:36.258075 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:36.258040    2579 patch_prober.go:28] interesting pod/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 14 03:16:36.258278 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:36.258090    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww" podUID="5de2715c-fdc1-4537-9bae-530e918e5ba5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 14 03:16:36.873982 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:36.873946    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:16:37.262871 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:37.262846    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:16:37.263864 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:37.263847    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:16:42.178511 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:42.178478    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-3a752f-29822595-dlmpm"]
Sep 14 03:16:42.182601 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:42.182575    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-3a752f-29822595-dlmpm"]
Sep 14 03:16:42.403713 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:42.403683    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd"]
Sep 14 03:16:42.412657 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:42.412630    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-3a752f-vgzpd"]
Sep 14 03:16:43.743925 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:43.743893    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a570fd4-4e1a-4ce2-a36b-1d8313991709" path="/var/lib/kubelet/pods/1a570fd4-4e1a-4ce2-a36b-1d8313991709/volumes"
Sep 14 03:16:43.744326 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:43.744234    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d170a489-d7ec-40e8-b6fc-426a564ced36" path="/var/lib/kubelet/pods/d170a489-d7ec-40e8-b6fc-426a564ced36/volumes"
Sep 14 03:16:46.149139 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:46.149113    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"
Sep 14 03:16:49.523984 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:49.523950    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8"]
Sep 14 03:16:49.526871 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:49.526855    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8"
Sep 14 03:16:49.537584 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:49.537567    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:16:49.545828 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:49.545803    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8"]
Sep 14 03:16:49.602365 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:49.602335    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1ead532-5619-48e4-8c8b-6c374209060e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8\" (UID: \"b1ead532-5619-48e4-8c8b-6c374209060e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8"
Sep 14 03:16:49.703710 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:49.703684    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1ead532-5619-48e4-8c8b-6c374209060e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8\" (UID: \"b1ead532-5619-48e4-8c8b-6c374209060e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8"
Sep 14 03:16:49.704424 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:49.704402    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1ead532-5619-48e4-8c8b-6c374209060e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8\" (UID: \"b1ead532-5619-48e4-8c8b-6c374209060e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8"
Sep 14 03:16:49.708066 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:49.708040    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"]
Sep 14 03:16:49.708264 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:49.708244    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk" podUID="f79d663e-c9e2-4040-8dfe-c21d73cd0198" containerName="istio-proxy" containerID="cri-o://c12c2e14f847912f41d92af17c0617630dd200960e9e673d91ee7538a4b597b9" gracePeriod=30
Sep 14 03:16:49.836956 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:49.836853    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8"
Sep 14 03:16:50.108315 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:50.108282    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8"]
Sep 14 03:16:50.109959 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:16:50.109931    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ead532_5619_48e4_8c8b_6c374209060e.slice/crio-bb6a60fa1554496a09fb14bab8ccbba4f58b65c2fd37a3694fa682310203a9be WatchSource:0}: Error finding container bb6a60fa1554496a09fb14bab8ccbba4f58b65c2fd37a3694fa682310203a9be: Status 404 returned error can't find the container with id bb6a60fa1554496a09fb14bab8ccbba4f58b65c2fd37a3694fa682310203a9be
Sep 14 03:16:50.232497 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:50.232467    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8" event={"ID":"b1ead532-5619-48e4-8c8b-6c374209060e","Type":"ContainerStarted","Data":"bb6a60fa1554496a09fb14bab8ccbba4f58b65c2fd37a3694fa682310203a9be"}
Sep 14 03:16:51.239960 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:51.239876    2579 generic.go:356] "Generic (PLEG): container finished" podID="b1ead532-5619-48e4-8c8b-6c374209060e" containerID="6479acef64be0fc1d2b74a4ef397c5ff691fc9af4c0dea0a3d5045b40288bd53" exitCode=0
Sep 14 03:16:51.239960 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:51.239912    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8" event={"ID":"b1ead532-5619-48e4-8c8b-6c374209060e","Type":"ContainerDied","Data":"6479acef64be0fc1d2b74a4ef397c5ff691fc9af4c0dea0a3d5045b40288bd53"}
Sep 14 03:16:52.399510 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:52.399482    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8"
Sep 14 03:16:52.531269 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:52.531199    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1ead532-5619-48e4-8c8b-6c374209060e-maas-api-service-ca\") pod \"b1ead532-5619-48e4-8c8b-6c374209060e\" (UID: \"b1ead532-5619-48e4-8c8b-6c374209060e\") "
Sep 14 03:16:52.531510 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:52.531488    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1ead532-5619-48e4-8c8b-6c374209060e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b1ead532-5619-48e4-8c8b-6c374209060e" (UID: "b1ead532-5619-48e4-8c8b-6c374209060e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:16:52.631840 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:52.631803    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1ead532-5619-48e4-8c8b-6c374209060e-maas-api-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:16:53.255954 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:53.255919    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8"
Sep 14 03:16:53.256160 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:53.255959    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8" event={"ID":"b1ead532-5619-48e4-8c8b-6c374209060e","Type":"ContainerDied","Data":"bb6a60fa1554496a09fb14bab8ccbba4f58b65c2fd37a3694fa682310203a9be"}
Sep 14 03:16:53.256160 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:53.256005    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb6a60fa1554496a09fb14bab8ccbba4f58b65c2fd37a3694fa682310203a9be"
Sep 14 03:16:54.732790 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:54.732765    2579 patch_prober.go:28] interesting pod/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 14 03:16:54.733076 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:54.732832    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk" podUID="f79d663e-c9e2-4040-8dfe-c21d73cd0198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 14 03:16:54.974123 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:54.974069    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:16:55.055405 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.055375    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-workload-certs\") pod \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") "
Sep 14 03:16:55.055568 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.055412    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-podinfo\") pod \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") "
Sep 14 03:16:55.055568 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.055458    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-token\") pod \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") "
Sep 14 03:16:55.055568 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.055478    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-workload-socket\") pod \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") "
Sep 14 03:16:55.055568 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.055531    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-credential-socket\") pod \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") "
Sep 14 03:16:55.055776 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.055601    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istiod-ca-cert\") pod \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") "
Sep 14 03:16:55.055776 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.055645    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-envoy\") pod \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") "
Sep 14 03:16:55.055776 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.055655    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f79d663e-c9e2-4040-8dfe-c21d73cd0198" (UID: "f79d663e-c9e2-4040-8dfe-c21d73cd0198"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:16:55.055776 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.055703    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wh2z\" (UniqueName: \"kubernetes.io/projected/f79d663e-c9e2-4040-8dfe-c21d73cd0198-kube-api-access-7wh2z\") pod \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") "
Sep 14 03:16:55.055776 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.055768    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-data\") pod \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\" (UID: \"f79d663e-c9e2-4040-8dfe-c21d73cd0198\") "
Sep 14 03:16:55.056007 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.055807    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f79d663e-c9e2-4040-8dfe-c21d73cd0198" (UID: "f79d663e-c9e2-4040-8dfe-c21d73cd0198"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:16:55.056007 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.055818    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f79d663e-c9e2-4040-8dfe-c21d73cd0198" (UID: "f79d663e-c9e2-4040-8dfe-c21d73cd0198"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:16:55.056007 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.055951    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f79d663e-c9e2-4040-8dfe-c21d73cd0198" (UID: "f79d663e-c9e2-4040-8dfe-c21d73cd0198"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:16:55.056150 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.056082    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-data" (OuterVolumeSpecName: "istio-data") pod "f79d663e-c9e2-4040-8dfe-c21d73cd0198" (UID: "f79d663e-c9e2-4040-8dfe-c21d73cd0198"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:16:55.056570 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.056469    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:16:55.056570 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.056493    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:16:55.056570 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.056508    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:16:55.056570 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.056522    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:16:55.056570 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.056535    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:16:55.057951 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.057913    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f79d663e-c9e2-4040-8dfe-c21d73cd0198" (UID: "f79d663e-c9e2-4040-8dfe-c21d73cd0198"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:16:55.058074 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.057942    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f79d663e-c9e2-4040-8dfe-c21d73cd0198-kube-api-access-7wh2z" (OuterVolumeSpecName: "kube-api-access-7wh2z") pod "f79d663e-c9e2-4040-8dfe-c21d73cd0198" (UID: "f79d663e-c9e2-4040-8dfe-c21d73cd0198"). InnerVolumeSpecName "kube-api-access-7wh2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:16:55.058074 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.058018    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f79d663e-c9e2-4040-8dfe-c21d73cd0198" (UID: "f79d663e-c9e2-4040-8dfe-c21d73cd0198"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:16:55.058163 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.058071    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-token" (OuterVolumeSpecName: "istio-token") pod "f79d663e-c9e2-4040-8dfe-c21d73cd0198" (UID: "f79d663e-c9e2-4040-8dfe-c21d73cd0198"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:16:55.157355 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.157331    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:16:55.157355 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.157356    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wh2z\" (UniqueName: \"kubernetes.io/projected/f79d663e-c9e2-4040-8dfe-c21d73cd0198-kube-api-access-7wh2z\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:16:55.157507 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.157366    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:16:55.157507 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.157376    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f79d663e-c9e2-4040-8dfe-c21d73cd0198-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:16:55.269666 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.269633    2579 generic.go:356] "Generic (PLEG): container finished" podID="f79d663e-c9e2-4040-8dfe-c21d73cd0198" containerID="c12c2e14f847912f41d92af17c0617630dd200960e9e673d91ee7538a4b597b9" exitCode=0
Sep 14 03:16:55.269825 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.269684    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"
Sep 14 03:16:55.269825 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.269700    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk" event={"ID":"f79d663e-c9e2-4040-8dfe-c21d73cd0198","Type":"ContainerDied","Data":"c12c2e14f847912f41d92af17c0617630dd200960e9e673d91ee7538a4b597b9"}
Sep 14 03:16:55.269825 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.269756    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk" event={"ID":"f79d663e-c9e2-4040-8dfe-c21d73cd0198","Type":"ContainerDied","Data":"3a55d35e50cd6d7d77a8c08ecb2f8c65672cb1f72bf2999a831869e2e44b2c4f"}
Sep 14 03:16:55.269825 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.269778    2579 scope.go:117] "RemoveContainer" containerID="c12c2e14f847912f41d92af17c0617630dd200960e9e673d91ee7538a4b597b9"
Sep 14 03:16:55.280285 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.280267    2579 scope.go:117] "RemoveContainer" containerID="7aee3806bdf0c82fd214dbf3cd012b0ebf9fecf3844e8005f86dd88fcc0e243a"
Sep 14 03:16:55.288826 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.288811    2579 scope.go:117] "RemoveContainer" containerID="c12c2e14f847912f41d92af17c0617630dd200960e9e673d91ee7538a4b597b9"
Sep 14 03:16:55.289092 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:16:55.289074    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c12c2e14f847912f41d92af17c0617630dd200960e9e673d91ee7538a4b597b9\": container with ID starting with c12c2e14f847912f41d92af17c0617630dd200960e9e673d91ee7538a4b597b9 not found: ID does not exist" containerID="c12c2e14f847912f41d92af17c0617630dd200960e9e673d91ee7538a4b597b9"
Sep 14 03:16:55.289149 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.289100    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c12c2e14f847912f41d92af17c0617630dd200960e9e673d91ee7538a4b597b9"} err="failed to get container status \"c12c2e14f847912f41d92af17c0617630dd200960e9e673d91ee7538a4b597b9\": rpc error: code = NotFound desc = could not find container \"c12c2e14f847912f41d92af17c0617630dd200960e9e673d91ee7538a4b597b9\": container with ID starting with c12c2e14f847912f41d92af17c0617630dd200960e9e673d91ee7538a4b597b9 not found: ID does not exist"
Sep 14 03:16:55.289149 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.289119    2579 scope.go:117] "RemoveContainer" containerID="7aee3806bdf0c82fd214dbf3cd012b0ebf9fecf3844e8005f86dd88fcc0e243a"
Sep 14 03:16:55.289404 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:16:55.289388    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aee3806bdf0c82fd214dbf3cd012b0ebf9fecf3844e8005f86dd88fcc0e243a\": container with ID starting with 7aee3806bdf0c82fd214dbf3cd012b0ebf9fecf3844e8005f86dd88fcc0e243a not found: ID does not exist" containerID="7aee3806bdf0c82fd214dbf3cd012b0ebf9fecf3844e8005f86dd88fcc0e243a"
Sep 14 03:16:55.289444 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.289409    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aee3806bdf0c82fd214dbf3cd012b0ebf9fecf3844e8005f86dd88fcc0e243a"} err="failed to get container status \"7aee3806bdf0c82fd214dbf3cd012b0ebf9fecf3844e8005f86dd88fcc0e243a\": rpc error: code = NotFound desc = could not find container \"7aee3806bdf0c82fd214dbf3cd012b0ebf9fecf3844e8005f86dd88fcc0e243a\": container with ID starting with 7aee3806bdf0c82fd214dbf3cd012b0ebf9fecf3844e8005f86dd88fcc0e243a not found: ID does not exist"
Sep 14 03:16:55.297362 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.297331    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"]
Sep 14 03:16:55.301585 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.301564    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-3a752f-openshift-default-7b87958d6d-lxzrk"]
Sep 14 03:16:55.744685 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:16:55.744656    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f79d663e-c9e2-4040-8dfe-c21d73cd0198" path="/var/lib/kubelet/pods/f79d663e-c9e2-4040-8dfe-c21d73cd0198/volumes"
Sep 14 03:17:05.507945 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:05.507913    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"]
Sep 14 03:17:05.508572 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:05.508134    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg" podUID="04c70921-127d-464b-9cad-49bef8598ffd" containerName="payload-processing" containerID="cri-o://d268d0b686d1913211687353acf2620e91f901e5eb78e073d5b121421787b9ce" gracePeriod=30
Sep 14 03:17:05.678680 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:05.678650    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8"]
Sep 14 03:17:05.680710 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:05.680678    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-3a33c1-pvfb8"]
Sep 14 03:17:05.745460 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:05.745433    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1ead532-5619-48e4-8c8b-6c374209060e" path="/var/lib/kubelet/pods/b1ead532-5619-48e4-8c8b-6c374209060e/volumes"
Sep 14 03:17:05.779994 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:05.779921    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"
Sep 14 03:17:05.968695 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:05.968663    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04c70921-127d-464b-9cad-49bef8598ffd-plugins-config-volume\") pod \"04c70921-127d-464b-9cad-49bef8598ffd\" (UID: \"04c70921-127d-464b-9cad-49bef8598ffd\") "
Sep 14 03:17:05.968851 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:05.968808    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdf7w\" (UniqueName: \"kubernetes.io/projected/04c70921-127d-464b-9cad-49bef8598ffd-kube-api-access-xdf7w\") pod \"04c70921-127d-464b-9cad-49bef8598ffd\" (UID: \"04c70921-127d-464b-9cad-49bef8598ffd\") "
Sep 14 03:17:05.968981 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:05.968958    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04c70921-127d-464b-9cad-49bef8598ffd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "04c70921-127d-464b-9cad-49bef8598ffd" (UID: "04c70921-127d-464b-9cad-49bef8598ffd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:05.969079 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:05.969067    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04c70921-127d-464b-9cad-49bef8598ffd-plugins-config-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:05.970983 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:05.970950    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04c70921-127d-464b-9cad-49bef8598ffd-kube-api-access-xdf7w" (OuterVolumeSpecName: "kube-api-access-xdf7w") pod "04c70921-127d-464b-9cad-49bef8598ffd" (UID: "04c70921-127d-464b-9cad-49bef8598ffd"). InnerVolumeSpecName "kube-api-access-xdf7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:06.070590 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:06.070522    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdf7w\" (UniqueName: \"kubernetes.io/projected/04c70921-127d-464b-9cad-49bef8598ffd-kube-api-access-xdf7w\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:06.340144 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:06.340066    2579 generic.go:356] "Generic (PLEG): container finished" podID="04c70921-127d-464b-9cad-49bef8598ffd" containerID="d268d0b686d1913211687353acf2620e91f901e5eb78e073d5b121421787b9ce" exitCode=0
Sep 14 03:17:06.340144 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:06.340107    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"
Sep 14 03:17:06.340144 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:06.340135    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg" event={"ID":"04c70921-127d-464b-9cad-49bef8598ffd","Type":"ContainerDied","Data":"d268d0b686d1913211687353acf2620e91f901e5eb78e073d5b121421787b9ce"}
Sep 14 03:17:06.340421 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:06.340194    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg" event={"ID":"04c70921-127d-464b-9cad-49bef8598ffd","Type":"ContainerDied","Data":"ae2059d69c98e23febfa64c0c1fc6147dc561e6dacb9524c7396b4dce31336d4"}
Sep 14 03:17:06.340421 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:06.340216    2579 scope.go:117] "RemoveContainer" containerID="d268d0b686d1913211687353acf2620e91f901e5eb78e073d5b121421787b9ce"
Sep 14 03:17:06.349867 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:06.349852    2579 scope.go:117] "RemoveContainer" containerID="d268d0b686d1913211687353acf2620e91f901e5eb78e073d5b121421787b9ce"
Sep 14 03:17:06.350114 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:17:06.350091    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d268d0b686d1913211687353acf2620e91f901e5eb78e073d5b121421787b9ce\": container with ID starting with d268d0b686d1913211687353acf2620e91f901e5eb78e073d5b121421787b9ce not found: ID does not exist" containerID="d268d0b686d1913211687353acf2620e91f901e5eb78e073d5b121421787b9ce"
Sep 14 03:17:06.350200 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:06.350122    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d268d0b686d1913211687353acf2620e91f901e5eb78e073d5b121421787b9ce"} err="failed to get container status \"d268d0b686d1913211687353acf2620e91f901e5eb78e073d5b121421787b9ce\": rpc error: code = NotFound desc = could not find container \"d268d0b686d1913211687353acf2620e91f901e5eb78e073d5b121421787b9ce\": container with ID starting with d268d0b686d1913211687353acf2620e91f901e5eb78e073d5b121421787b9ce not found: ID does not exist"
Sep 14 03:17:06.365076 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:06.365056    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"]
Sep 14 03:17:06.371969 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:06.371950    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-3a33c1-8f9b5d4d7-jwbkg"]
Sep 14 03:17:07.749636 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:07.749592    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04c70921-127d-464b-9cad-49bef8598ffd" path="/var/lib/kubelet/pods/04c70921-127d-464b-9cad-49bef8598ffd/volumes"
Sep 14 03:17:13.310239 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.310211    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"]
Sep 14 03:17:13.310625 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.310428    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww" podUID="5de2715c-fdc1-4537-9bae-530e918e5ba5" containerName="istio-proxy" containerID="cri-o://9430259db73b2fb81bad80b1564065df9275e42652e17680dce89aabf7b9cea5" gracePeriod=30
Sep 14 03:17:13.679057 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.678968    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97da78-6sr8c"]
Sep 14 03:17:13.679499 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.679481    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f79d663e-c9e2-4040-8dfe-c21d73cd0198" containerName="istio-proxy"
Sep 14 03:17:13.679598 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.679501    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f79d663e-c9e2-4040-8dfe-c21d73cd0198" containerName="istio-proxy"
Sep 14 03:17:13.679598 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.679520    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1ead532-5619-48e4-8c8b-6c374209060e" containerName="revoke-keys"
Sep 14 03:17:13.679598 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.679528    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1ead532-5619-48e4-8c8b-6c374209060e" containerName="revoke-keys"
Sep 14 03:17:13.679598 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.679545    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f79d663e-c9e2-4040-8dfe-c21d73cd0198" containerName="istio-proxy"
Sep 14 03:17:13.679598 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.679553    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f79d663e-c9e2-4040-8dfe-c21d73cd0198" containerName="istio-proxy"
Sep 14 03:17:13.679598 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.679565    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04c70921-127d-464b-9cad-49bef8598ffd" containerName="payload-processing"
Sep 14 03:17:13.679598 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.679574    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="04c70921-127d-464b-9cad-49bef8598ffd" containerName="payload-processing"
Sep 14 03:17:13.679933 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.679672    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1ead532-5619-48e4-8c8b-6c374209060e" containerName="revoke-keys"
Sep 14 03:17:13.679933 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.679686    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f79d663e-c9e2-4040-8dfe-c21d73cd0198" containerName="istio-proxy"
Sep 14 03:17:13.679933 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.679698    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="04c70921-127d-464b-9cad-49bef8598ffd" containerName="payload-processing"
Sep 14 03:17:13.679933 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.679708    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f79d663e-c9e2-4040-8dfe-c21d73cd0198" containerName="istio-proxy"
Sep 14 03:17:13.683768 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.683748    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97da78-6sr8c"
Sep 14 03:17:13.687519 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.687498    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:17:13.695828 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.695794    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97da78-6sr8c"]
Sep 14 03:17:13.742902 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.742867    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67e6e321-a7c2-4c44-9f17-bc32b6c31148-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-97da78-6sr8c\" (UID: \"67e6e321-a7c2-4c44-9f17-bc32b6c31148\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97da78-6sr8c"
Sep 14 03:17:13.843879 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.843847    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67e6e321-a7c2-4c44-9f17-bc32b6c31148-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-97da78-6sr8c\" (UID: \"67e6e321-a7c2-4c44-9f17-bc32b6c31148\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97da78-6sr8c"
Sep 14 03:17:13.844500 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.844473    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67e6e321-a7c2-4c44-9f17-bc32b6c31148-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-97da78-6sr8c\" (UID: \"67e6e321-a7c2-4c44-9f17-bc32b6c31148\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97da78-6sr8c"
Sep 14 03:17:13.995602 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:13.995570    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97da78-6sr8c"
Sep 14 03:17:14.275763 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:14.275738    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97da78-6sr8c"]
Sep 14 03:17:14.278667 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:17:14.278641    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e6e321_a7c2_4c44_9f17_bc32b6c31148.slice/crio-aee44b46f15c6ba996f298e3e75eb5a716f0cb78da4be7483fb17ad242a84890 WatchSource:0}: Error finding container aee44b46f15c6ba996f298e3e75eb5a716f0cb78da4be7483fb17ad242a84890: Status 404 returned error can't find the container with id aee44b46f15c6ba996f298e3e75eb5a716f0cb78da4be7483fb17ad242a84890
Sep 14 03:17:14.392004 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:14.391961    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97da78-6sr8c" event={"ID":"67e6e321-a7c2-4c44-9f17-bc32b6c31148","Type":"ContainerStarted","Data":"aee44b46f15c6ba996f298e3e75eb5a716f0cb78da4be7483fb17ad242a84890"}
Sep 14 03:17:15.399898 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:15.399864    2579 generic.go:356] "Generic (PLEG): container finished" podID="67e6e321-a7c2-4c44-9f17-bc32b6c31148" containerID="7c3be39f961d73e4eed663d1411c942fa94852f14497e7ed6830d6cb9bbad708" exitCode=0
Sep 14 03:17:15.400336 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:15.399911    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97da78-6sr8c" event={"ID":"67e6e321-a7c2-4c44-9f17-bc32b6c31148","Type":"ContainerDied","Data":"7c3be39f961d73e4eed663d1411c942fa94852f14497e7ed6830d6cb9bbad708"}
Sep 14 03:17:16.551225 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:16.551202    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97da78-6sr8c"
Sep 14 03:17:16.672017 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:16.671931    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67e6e321-a7c2-4c44-9f17-bc32b6c31148-maas-api-service-ca\") pod \"67e6e321-a7c2-4c44-9f17-bc32b6c31148\" (UID: \"67e6e321-a7c2-4c44-9f17-bc32b6c31148\") "
Sep 14 03:17:16.672277 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:16.672254    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67e6e321-a7c2-4c44-9f17-bc32b6c31148-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "67e6e321-a7c2-4c44-9f17-bc32b6c31148" (UID: "67e6e321-a7c2-4c44-9f17-bc32b6c31148"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:16.773610 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:16.773581    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67e6e321-a7c2-4c44-9f17-bc32b6c31148-maas-api-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:17.412148 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:17.412122    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97da78-6sr8c"
Sep 14 03:17:17.412405 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:17.412119    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97da78-6sr8c" event={"ID":"67e6e321-a7c2-4c44-9f17-bc32b6c31148","Type":"ContainerDied","Data":"aee44b46f15c6ba996f298e3e75eb5a716f0cb78da4be7483fb17ad242a84890"}
Sep 14 03:17:17.412405 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:17.412258    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aee44b46f15c6ba996f298e3e75eb5a716f0cb78da4be7483fb17ad242a84890"
Sep 14 03:17:17.938402 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:17.938372    2579 scope.go:117] "RemoveContainer" containerID="c35b338c563948abe87736abd63795c205b9116a435f9f36c07f8885fb4f73f6"
Sep 14 03:17:18.565772 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.565749    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:17:18.689889 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.689805    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-credential-socket\") pod \"5de2715c-fdc1-4537-9bae-530e918e5ba5\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") "
Sep 14 03:17:18.689889 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.689851    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-workload-certs\") pod \"5de2715c-fdc1-4537-9bae-530e918e5ba5\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") "
Sep 14 03:17:18.689889 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.689887    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-data\") pod \"5de2715c-fdc1-4537-9bae-530e918e5ba5\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") "
Sep 14 03:17:18.690197 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.689904    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-podinfo\") pod \"5de2715c-fdc1-4537-9bae-530e918e5ba5\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") "
Sep 14 03:17:18.690197 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.689953    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-workload-socket\") pod \"5de2715c-fdc1-4537-9bae-530e918e5ba5\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") "
Sep 14 03:17:18.690197 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.689971    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-envoy\") pod \"5de2715c-fdc1-4537-9bae-530e918e5ba5\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") "
Sep 14 03:17:18.690197 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.690006    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzfwh\" (UniqueName: \"kubernetes.io/projected/5de2715c-fdc1-4537-9bae-530e918e5ba5-kube-api-access-wzfwh\") pod \"5de2715c-fdc1-4537-9bae-530e918e5ba5\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") "
Sep 14 03:17:18.690197 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.690050    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5de2715c-fdc1-4537-9bae-530e918e5ba5-istiod-ca-cert\") pod \"5de2715c-fdc1-4537-9bae-530e918e5ba5\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") "
Sep 14 03:17:18.690197 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.690095    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5de2715c-fdc1-4537-9bae-530e918e5ba5" (UID: "5de2715c-fdc1-4537-9bae-530e918e5ba5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:18.690197 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.690113    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-token\") pod \"5de2715c-fdc1-4537-9bae-530e918e5ba5\" (UID: \"5de2715c-fdc1-4537-9bae-530e918e5ba5\") "
Sep 14 03:17:18.690493 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.690226    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-data" (OuterVolumeSpecName: "istio-data") pod "5de2715c-fdc1-4537-9bae-530e918e5ba5" (UID: "5de2715c-fdc1-4537-9bae-530e918e5ba5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:18.690493 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.690245    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5de2715c-fdc1-4537-9bae-530e918e5ba5" (UID: "5de2715c-fdc1-4537-9bae-530e918e5ba5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:18.690493 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.690258    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5de2715c-fdc1-4537-9bae-530e918e5ba5" (UID: "5de2715c-fdc1-4537-9bae-530e918e5ba5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:18.690642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.690515    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:18.690642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.690535    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:18.690642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.690547    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:18.690642 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.690560    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:18.691010 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.690648    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5de2715c-fdc1-4537-9bae-530e918e5ba5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5de2715c-fdc1-4537-9bae-530e918e5ba5" (UID: "5de2715c-fdc1-4537-9bae-530e918e5ba5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:18.692460 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.692436    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-token" (OuterVolumeSpecName: "istio-token") pod "5de2715c-fdc1-4537-9bae-530e918e5ba5" (UID: "5de2715c-fdc1-4537-9bae-530e918e5ba5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:18.692650 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.692625    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5de2715c-fdc1-4537-9bae-530e918e5ba5" (UID: "5de2715c-fdc1-4537-9bae-530e918e5ba5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:17:18.692738 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.692715    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5de2715c-fdc1-4537-9bae-530e918e5ba5" (UID: "5de2715c-fdc1-4537-9bae-530e918e5ba5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:18.692812 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.692792    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5de2715c-fdc1-4537-9bae-530e918e5ba5-kube-api-access-wzfwh" (OuterVolumeSpecName: "kube-api-access-wzfwh") pod "5de2715c-fdc1-4537-9bae-530e918e5ba5" (UID: "5de2715c-fdc1-4537-9bae-530e918e5ba5"). InnerVolumeSpecName "kube-api-access-wzfwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:18.791271 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.791240    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:18.791271 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.791268    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:18.791481 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.791281    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5de2715c-fdc1-4537-9bae-530e918e5ba5-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:18.791481 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.791296    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzfwh\" (UniqueName: \"kubernetes.io/projected/5de2715c-fdc1-4537-9bae-530e918e5ba5-kube-api-access-wzfwh\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:18.791481 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:18.791310    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5de2715c-fdc1-4537-9bae-530e918e5ba5-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:19.424913 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:19.424877    2579 generic.go:356] "Generic (PLEG): container finished" podID="5de2715c-fdc1-4537-9bae-530e918e5ba5" containerID="9430259db73b2fb81bad80b1564065df9275e42652e17680dce89aabf7b9cea5" exitCode=0
Sep 14 03:17:19.425418 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:19.424938    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"
Sep 14 03:17:19.425418 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:19.424952    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww" event={"ID":"5de2715c-fdc1-4537-9bae-530e918e5ba5","Type":"ContainerDied","Data":"9430259db73b2fb81bad80b1564065df9275e42652e17680dce89aabf7b9cea5"}
Sep 14 03:17:19.425418 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:19.425003    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww" event={"ID":"5de2715c-fdc1-4537-9bae-530e918e5ba5","Type":"ContainerDied","Data":"50b07c4b2685585c49ce34a5449450b4f72c263c3dd9c14772e1b6e4ba81299f"}
Sep 14 03:17:19.425418 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:19.425025    2579 scope.go:117] "RemoveContainer" containerID="9430259db73b2fb81bad80b1564065df9275e42652e17680dce89aabf7b9cea5"
Sep 14 03:17:19.434320 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:19.434305    2579 scope.go:117] "RemoveContainer" containerID="140b7181ddb27978ca0765dcac6f2d8bf83143db42d7842270e4ee5e2c13a26e"
Sep 14 03:17:19.442127 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:19.442112    2579 scope.go:117] "RemoveContainer" containerID="9430259db73b2fb81bad80b1564065df9275e42652e17680dce89aabf7b9cea5"
Sep 14 03:17:19.442432 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:17:19.442410    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9430259db73b2fb81bad80b1564065df9275e42652e17680dce89aabf7b9cea5\": container with ID starting with 9430259db73b2fb81bad80b1564065df9275e42652e17680dce89aabf7b9cea5 not found: ID does not exist" containerID="9430259db73b2fb81bad80b1564065df9275e42652e17680dce89aabf7b9cea5"
Sep 14 03:17:19.442487 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:19.442438    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9430259db73b2fb81bad80b1564065df9275e42652e17680dce89aabf7b9cea5"} err="failed to get container status \"9430259db73b2fb81bad80b1564065df9275e42652e17680dce89aabf7b9cea5\": rpc error: code = NotFound desc = could not find container \"9430259db73b2fb81bad80b1564065df9275e42652e17680dce89aabf7b9cea5\": container with ID starting with 9430259db73b2fb81bad80b1564065df9275e42652e17680dce89aabf7b9cea5 not found: ID does not exist"
Sep 14 03:17:19.442487 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:19.442454    2579 scope.go:117] "RemoveContainer" containerID="140b7181ddb27978ca0765dcac6f2d8bf83143db42d7842270e4ee5e2c13a26e"
Sep 14 03:17:19.442692 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:17:19.442678    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"140b7181ddb27978ca0765dcac6f2d8bf83143db42d7842270e4ee5e2c13a26e\": container with ID starting with 140b7181ddb27978ca0765dcac6f2d8bf83143db42d7842270e4ee5e2c13a26e not found: ID does not exist" containerID="140b7181ddb27978ca0765dcac6f2d8bf83143db42d7842270e4ee5e2c13a26e"
Sep 14 03:17:19.442730 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:19.442697    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"140b7181ddb27978ca0765dcac6f2d8bf83143db42d7842270e4ee5e2c13a26e"} err="failed to get container status \"140b7181ddb27978ca0765dcac6f2d8bf83143db42d7842270e4ee5e2c13a26e\": rpc error: code = NotFound desc = could not find container \"140b7181ddb27978ca0765dcac6f2d8bf83143db42d7842270e4ee5e2c13a26e\": container with ID starting with 140b7181ddb27978ca0765dcac6f2d8bf83143db42d7842270e4ee5e2c13a26e not found: ID does not exist"
Sep 14 03:17:19.447551 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:19.447529    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"]
Sep 14 03:17:19.453605 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:19.453587    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-3a33c1-openshift-default-5bfb98f9cb-9hfww"]
Sep 14 03:17:19.744502 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:19.744473    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5de2715c-fdc1-4537-9bae-530e918e5ba5" path="/var/lib/kubelet/pods/5de2715c-fdc1-4537-9bae-530e918e5ba5/volumes"
Sep 14 03:17:29.350776 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:29.350744    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"]
Sep 14 03:17:29.351287 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:29.351015    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95" podUID="493ba152-460b-4780-ae56-e56bbc62ad40" containerName="payload-pre-processing" containerID="cri-o://1cec149c0502a82161ac6fc8018bb98b48485e64dbef812223db5ab0a94b27e1" gracePeriod=30
Sep 14 03:17:29.485760 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:29.485722    2579 generic.go:356] "Generic (PLEG): container finished" podID="493ba152-460b-4780-ae56-e56bbc62ad40" containerID="1cec149c0502a82161ac6fc8018bb98b48485e64dbef812223db5ab0a94b27e1" exitCode=0
Sep 14 03:17:29.485901 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:29.485801    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95" event={"ID":"493ba152-460b-4780-ae56-e56bbc62ad40","Type":"ContainerDied","Data":"1cec149c0502a82161ac6fc8018bb98b48485e64dbef812223db5ab0a94b27e1"}
Sep 14 03:17:29.521507 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:29.521286    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97da78-6sr8c"]
Sep 14 03:17:29.523659 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:29.523638    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-97da78-6sr8c"]
Sep 14 03:17:29.606904 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:29.606850    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"
Sep 14 03:17:29.691454 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:29.691421    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/493ba152-460b-4780-ae56-e56bbc62ad40-plugins-config-volume\") pod \"493ba152-460b-4780-ae56-e56bbc62ad40\" (UID: \"493ba152-460b-4780-ae56-e56bbc62ad40\") "
Sep 14 03:17:29.691660 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:29.691486    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shw44\" (UniqueName: \"kubernetes.io/projected/493ba152-460b-4780-ae56-e56bbc62ad40-kube-api-access-shw44\") pod \"493ba152-460b-4780-ae56-e56bbc62ad40\" (UID: \"493ba152-460b-4780-ae56-e56bbc62ad40\") "
Sep 14 03:17:29.692226 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:29.692197    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/493ba152-460b-4780-ae56-e56bbc62ad40-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "493ba152-460b-4780-ae56-e56bbc62ad40" (UID: "493ba152-460b-4780-ae56-e56bbc62ad40"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:29.697580 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:29.696703    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/493ba152-460b-4780-ae56-e56bbc62ad40-kube-api-access-shw44" (OuterVolumeSpecName: "kube-api-access-shw44") pod "493ba152-460b-4780-ae56-e56bbc62ad40" (UID: "493ba152-460b-4780-ae56-e56bbc62ad40"). InnerVolumeSpecName "kube-api-access-shw44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:29.749951 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:29.749917    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67e6e321-a7c2-4c44-9f17-bc32b6c31148" path="/var/lib/kubelet/pods/67e6e321-a7c2-4c44-9f17-bc32b6c31148/volumes"
Sep 14 03:17:29.793005 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:29.792850    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/493ba152-460b-4780-ae56-e56bbc62ad40-plugins-config-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:29.793005 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:29.792884    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shw44\" (UniqueName: \"kubernetes.io/projected/493ba152-460b-4780-ae56-e56bbc62ad40-kube-api-access-shw44\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:30.494279 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:30.494250    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"
Sep 14 03:17:30.494279 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:30.494263    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95" event={"ID":"493ba152-460b-4780-ae56-e56bbc62ad40","Type":"ContainerDied","Data":"dfb5263be6f26efae62a2fda54484a06ae43e8b6b2e06d002d77421046d7cba4"}
Sep 14 03:17:30.494804 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:30.494337    2579 scope.go:117] "RemoveContainer" containerID="1cec149c0502a82161ac6fc8018bb98b48485e64dbef812223db5ab0a94b27e1"
Sep 14 03:17:30.514954 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:30.514933    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"]
Sep 14 03:17:30.518596 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:30.518577    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-97da78-75ffd978bb-9hd95"]
Sep 14 03:17:31.743274 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:31.743246    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="493ba152-460b-4780-ae56-e56bbc62ad40" path="/var/lib/kubelet/pods/493ba152-460b-4780-ae56-e56bbc62ad40/volumes"
Sep 14 03:17:36.544132 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.544099    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66"]
Sep 14 03:17:36.544601 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.544528    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5de2715c-fdc1-4537-9bae-530e918e5ba5" containerName="istio-proxy"
Sep 14 03:17:36.544601 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.544541    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5de2715c-fdc1-4537-9bae-530e918e5ba5" containerName="istio-proxy"
Sep 14 03:17:36.544601 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.544550    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67e6e321-a7c2-4c44-9f17-bc32b6c31148" containerName="revoke-keys"
Sep 14 03:17:36.544601 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.544556    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="67e6e321-a7c2-4c44-9f17-bc32b6c31148" containerName="revoke-keys"
Sep 14 03:17:36.544601 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.544567    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5de2715c-fdc1-4537-9bae-530e918e5ba5" containerName="istio-proxy"
Sep 14 03:17:36.544601 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.544573    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5de2715c-fdc1-4537-9bae-530e918e5ba5" containerName="istio-proxy"
Sep 14 03:17:36.544601 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.544581    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="493ba152-460b-4780-ae56-e56bbc62ad40" containerName="payload-pre-processing"
Sep 14 03:17:36.544601 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.544586    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="493ba152-460b-4780-ae56-e56bbc62ad40" containerName="payload-pre-processing"
Sep 14 03:17:36.544869 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.544648    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="67e6e321-a7c2-4c44-9f17-bc32b6c31148" containerName="revoke-keys"
Sep 14 03:17:36.544869 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.544656    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="493ba152-460b-4780-ae56-e56bbc62ad40" containerName="payload-pre-processing"
Sep 14 03:17:36.544869 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.544665    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5de2715c-fdc1-4537-9bae-530e918e5ba5" containerName="istio-proxy"
Sep 14 03:17:36.544869 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.544671    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5de2715c-fdc1-4537-9bae-530e918e5ba5" containerName="istio-proxy"
Sep 14 03:17:36.549202 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.549166    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66"
Sep 14 03:17:36.551884 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.551859    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:17:36.557716 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.557696    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66"]
Sep 14 03:17:36.662009 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.661977    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a9d1075-a976-4a78-b59c-9812361ccd1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66\" (UID: \"7a9d1075-a976-4a78-b59c-9812361ccd1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66"
Sep 14 03:17:36.762691 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.762660    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a9d1075-a976-4a78-b59c-9812361ccd1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66\" (UID: \"7a9d1075-a976-4a78-b59c-9812361ccd1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66"
Sep 14 03:17:36.763318 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.763299    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a9d1075-a976-4a78-b59c-9812361ccd1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66\" (UID: \"7a9d1075-a976-4a78-b59c-9812361ccd1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66"
Sep 14 03:17:36.862085 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:36.862007    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66"
Sep 14 03:17:37.141163 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:37.141139    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66"]
Sep 14 03:17:37.143046 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:17:37.143019    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a9d1075_a976_4a78_b59c_9812361ccd1f.slice/crio-a971e4e09b5be0b6ac8e838414cb2240971421c130713407c5d79972f92893ff WatchSource:0}: Error finding container a971e4e09b5be0b6ac8e838414cb2240971421c130713407c5d79972f92893ff: Status 404 returned error can't find the container with id a971e4e09b5be0b6ac8e838414cb2240971421c130713407c5d79972f92893ff
Sep 14 03:17:37.538274 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:37.538240    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66" event={"ID":"7a9d1075-a976-4a78-b59c-9812361ccd1f","Type":"ContainerStarted","Data":"d0c15079c41f9a0581b845c5559bd3b869b27d3704687b76777be29ff04550ae"}
Sep 14 03:17:37.538436 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:37.538290    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66" event={"ID":"7a9d1075-a976-4a78-b59c-9812361ccd1f","Type":"ContainerStarted","Data":"a971e4e09b5be0b6ac8e838414cb2240971421c130713407c5d79972f92893ff"}
Sep 14 03:17:38.545727 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:38.545695    2579 generic.go:356] "Generic (PLEG): container finished" podID="7a9d1075-a976-4a78-b59c-9812361ccd1f" containerID="d0c15079c41f9a0581b845c5559bd3b869b27d3704687b76777be29ff04550ae" exitCode=0
Sep 14 03:17:38.546119 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:38.545777    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66" event={"ID":"7a9d1075-a976-4a78-b59c-9812361ccd1f","Type":"ContainerDied","Data":"d0c15079c41f9a0581b845c5559bd3b869b27d3704687b76777be29ff04550ae"}
Sep 14 03:17:39.687982 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:39.687961    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66"
Sep 14 03:17:39.791691 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:39.791657    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a9d1075-a976-4a78-b59c-9812361ccd1f-maas-api-service-ca\") pod \"7a9d1075-a976-4a78-b59c-9812361ccd1f\" (UID: \"7a9d1075-a976-4a78-b59c-9812361ccd1f\") "
Sep 14 03:17:39.792051 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:39.792027    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a9d1075-a976-4a78-b59c-9812361ccd1f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7a9d1075-a976-4a78-b59c-9812361ccd1f" (UID: "7a9d1075-a976-4a78-b59c-9812361ccd1f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:39.893328 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:39.893247    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a9d1075-a976-4a78-b59c-9812361ccd1f-maas-api-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:40.558008 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:40.557979    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66"
Sep 14 03:17:40.558256 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:40.558017    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66" event={"ID":"7a9d1075-a976-4a78-b59c-9812361ccd1f","Type":"ContainerDied","Data":"a971e4e09b5be0b6ac8e838414cb2240971421c130713407c5d79972f92893ff"}
Sep 14 03:17:40.558256 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:40.558075    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a971e4e09b5be0b6ac8e838414cb2240971421c130713407c5d79972f92893ff"
Sep 14 03:17:44.454384 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:44.454347    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn"]
Sep 14 03:17:44.454951 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:44.454931    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a9d1075-a976-4a78-b59c-9812361ccd1f" containerName="revoke-keys"
Sep 14 03:17:44.455038 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:44.454954    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a9d1075-a976-4a78-b59c-9812361ccd1f" containerName="revoke-keys"
Sep 14 03:17:44.455102 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:44.455050    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a9d1075-a976-4a78-b59c-9812361ccd1f" containerName="revoke-keys"
Sep 14 03:17:44.458191 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:44.458157    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn"
Sep 14 03:17:44.461544 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:44.461521    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:17:44.468675 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:44.468649    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn"]
Sep 14 03:17:44.644744 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:44.644714    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c017b1f-7231-4672-8106-062df68418a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn\" (UID: \"8c017b1f-7231-4672-8106-062df68418a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn"
Sep 14 03:17:44.745786 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:44.745747    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c017b1f-7231-4672-8106-062df68418a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn\" (UID: \"8c017b1f-7231-4672-8106-062df68418a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn"
Sep 14 03:17:44.746369 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:44.746349    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c017b1f-7231-4672-8106-062df68418a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn\" (UID: \"8c017b1f-7231-4672-8106-062df68418a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn"
Sep 14 03:17:44.769313 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:44.769292    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn"
Sep 14 03:17:45.055730 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:45.055655    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn"]
Sep 14 03:17:45.059188 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:17:45.059146    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c017b1f_7231_4672_8106_062df68418a7.slice/crio-1c345215b67fdcd5fd73397c4d980fc9811ac3a93b6236122c218dbabfbc29ea WatchSource:0}: Error finding container 1c345215b67fdcd5fd73397c4d980fc9811ac3a93b6236122c218dbabfbc29ea: Status 404 returned error can't find the container with id 1c345215b67fdcd5fd73397c4d980fc9811ac3a93b6236122c218dbabfbc29ea
Sep 14 03:17:45.589075 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:45.589035    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn" event={"ID":"8c017b1f-7231-4672-8106-062df68418a7","Type":"ContainerStarted","Data":"4c1f158de80b497fdf52f93fe02549eed9706debeded0c1a62fbfe7e5c413ecf"}
Sep 14 03:17:45.589467 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:45.589100    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn" event={"ID":"8c017b1f-7231-4672-8106-062df68418a7","Type":"ContainerStarted","Data":"1c345215b67fdcd5fd73397c4d980fc9811ac3a93b6236122c218dbabfbc29ea"}
Sep 14 03:17:45.637924 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:45.637863    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn" podStartSLOduration=1.6378470410000001 podStartE2EDuration="1.637847041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:17:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:17:45.637834361 +0000 UTC m=+1528.423896274" watchObservedRunningTime="2026-09-14 03:17:45.637847041 +0000 UTC m=+1528.423908954"
Sep 14 03:17:46.596605 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:46.596574    2579 generic.go:356] "Generic (PLEG): container finished" podID="8c017b1f-7231-4672-8106-062df68418a7" containerID="4c1f158de80b497fdf52f93fe02549eed9706debeded0c1a62fbfe7e5c413ecf" exitCode=0
Sep 14 03:17:46.596975 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:46.596672    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn" event={"ID":"8c017b1f-7231-4672-8106-062df68418a7","Type":"ContainerDied","Data":"4c1f158de80b497fdf52f93fe02549eed9706debeded0c1a62fbfe7e5c413ecf"}
Sep 14 03:17:47.037612 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.037576    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ce5b9b-29822595-h6kjt"]
Sep 14 03:17:47.042109 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.042084    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-ce5b9b-29822595-h6kjt"]
Sep 14 03:17:47.142402 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.142373    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"]
Sep 14 03:17:47.142588 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.142563    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br" podUID="c55cf6b4-ff06-406e-b64b-548320256283" containerName="payload-processing" containerID="cri-o://98ff40ec7de6cbcb80aa4709c8608edbb99069e8d37ff3684078d930b980078c" gracePeriod=30
Sep 14 03:17:47.415730 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.415703    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"
Sep 14 03:17:47.573480 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.573395    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkpbx\" (UniqueName: \"kubernetes.io/projected/c55cf6b4-ff06-406e-b64b-548320256283-kube-api-access-gkpbx\") pod \"c55cf6b4-ff06-406e-b64b-548320256283\" (UID: \"c55cf6b4-ff06-406e-b64b-548320256283\") "
Sep 14 03:17:47.573480 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.573438    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c55cf6b4-ff06-406e-b64b-548320256283-plugins-config-volume\") pod \"c55cf6b4-ff06-406e-b64b-548320256283\" (UID: \"c55cf6b4-ff06-406e-b64b-548320256283\") "
Sep 14 03:17:47.573845 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.573818    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c55cf6b4-ff06-406e-b64b-548320256283-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c55cf6b4-ff06-406e-b64b-548320256283" (UID: "c55cf6b4-ff06-406e-b64b-548320256283"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:47.575757 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.575725    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c55cf6b4-ff06-406e-b64b-548320256283-kube-api-access-gkpbx" (OuterVolumeSpecName: "kube-api-access-gkpbx") pod "c55cf6b4-ff06-406e-b64b-548320256283" (UID: "c55cf6b4-ff06-406e-b64b-548320256283"). InnerVolumeSpecName "kube-api-access-gkpbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:47.604163 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.604130    2579 generic.go:356] "Generic (PLEG): container finished" podID="c55cf6b4-ff06-406e-b64b-548320256283" containerID="98ff40ec7de6cbcb80aa4709c8608edbb99069e8d37ff3684078d930b980078c" exitCode=0
Sep 14 03:17:47.604493 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.604185    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"
Sep 14 03:17:47.604493 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.604237    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br" event={"ID":"c55cf6b4-ff06-406e-b64b-548320256283","Type":"ContainerDied","Data":"98ff40ec7de6cbcb80aa4709c8608edbb99069e8d37ff3684078d930b980078c"}
Sep 14 03:17:47.604493 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.604295    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br" event={"ID":"c55cf6b4-ff06-406e-b64b-548320256283","Type":"ContainerDied","Data":"905fe7a72f05fce0b1645e10c2c46d2d8d0deee0c562bb165f8d7d774ec21468"}
Sep 14 03:17:47.604493 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.604322    2579 scope.go:117] "RemoveContainer" containerID="98ff40ec7de6cbcb80aa4709c8608edbb99069e8d37ff3684078d930b980078c"
Sep 14 03:17:47.613715 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.613700    2579 scope.go:117] "RemoveContainer" containerID="98ff40ec7de6cbcb80aa4709c8608edbb99069e8d37ff3684078d930b980078c"
Sep 14 03:17:47.614069 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:17:47.614051    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98ff40ec7de6cbcb80aa4709c8608edbb99069e8d37ff3684078d930b980078c\": container with ID starting with 98ff40ec7de6cbcb80aa4709c8608edbb99069e8d37ff3684078d930b980078c not found: ID does not exist" containerID="98ff40ec7de6cbcb80aa4709c8608edbb99069e8d37ff3684078d930b980078c"
Sep 14 03:17:47.614136 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.614078    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98ff40ec7de6cbcb80aa4709c8608edbb99069e8d37ff3684078d930b980078c"} err="failed to get container status \"98ff40ec7de6cbcb80aa4709c8608edbb99069e8d37ff3684078d930b980078c\": rpc error: code = NotFound desc = could not find container \"98ff40ec7de6cbcb80aa4709c8608edbb99069e8d37ff3684078d930b980078c\": container with ID starting with 98ff40ec7de6cbcb80aa4709c8608edbb99069e8d37ff3684078d930b980078c not found: ID does not exist"
Sep 14 03:17:47.630926 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.630831    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"]
Sep 14 03:17:47.634727 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.634705    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-ce5b9b-fbc675bc7-xz9br"]
Sep 14 03:17:47.675776 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.675738    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkpbx\" (UniqueName: \"kubernetes.io/projected/c55cf6b4-ff06-406e-b64b-548320256283-kube-api-access-gkpbx\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:47.676028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.676007    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c55cf6b4-ff06-406e-b64b-548320256283-plugins-config-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:47.685753 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.685726    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66"]
Sep 14 03:17:47.687914 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.687891    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-ce5b9b-f7j66"]
Sep 14 03:17:47.736656 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.736637    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn"
Sep 14 03:17:47.743678 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.743647    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a9d1075-a976-4a78-b59c-9812361ccd1f" path="/var/lib/kubelet/pods/7a9d1075-a976-4a78-b59c-9812361ccd1f/volumes"
Sep 14 03:17:47.744019 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.743991    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82a1f589-deb7-410d-8627-2bb120c63c23" path="/var/lib/kubelet/pods/82a1f589-deb7-410d-8627-2bb120c63c23/volumes"
Sep 14 03:17:47.744364 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.744346    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c55cf6b4-ff06-406e-b64b-548320256283" path="/var/lib/kubelet/pods/c55cf6b4-ff06-406e-b64b-548320256283/volumes"
Sep 14 03:17:47.783432 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.783405    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c017b1f-7231-4672-8106-062df68418a7-maas-api-service-ca\") pod \"8c017b1f-7231-4672-8106-062df68418a7\" (UID: \"8c017b1f-7231-4672-8106-062df68418a7\") "
Sep 14 03:17:47.783750 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.783723    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c017b1f-7231-4672-8106-062df68418a7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8c017b1f-7231-4672-8106-062df68418a7" (UID: "8c017b1f-7231-4672-8106-062df68418a7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:47.884365 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:47.884286    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c017b1f-7231-4672-8106-062df68418a7-maas-api-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:48.611365 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:48.611335    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn"
Sep 14 03:17:48.611365 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:48.611346    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn" event={"ID":"8c017b1f-7231-4672-8106-062df68418a7","Type":"ContainerDied","Data":"1c345215b67fdcd5fd73397c4d980fc9811ac3a93b6236122c218dbabfbc29ea"}
Sep 14 03:17:48.611815 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:48.611394    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c345215b67fdcd5fd73397c4d980fc9811ac3a93b6236122c218dbabfbc29ea"
Sep 14 03:17:49.346536 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:49.346500    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"]
Sep 14 03:17:54.662783 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:54.662753    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"]
Sep 14 03:17:54.663245 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:54.663005    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9" podUID="bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" containerName="istio-proxy" containerID="cri-o://c9ad893fe4d4d8cfe7de824e9125d0bb527b666501e661df49c5854004ec9539" gracePeriod=30
Sep 14 03:17:55.025526 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:55.025494    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9"]
Sep 14 03:17:55.027038 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:55.027000    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c55cf6b4-ff06-406e-b64b-548320256283" containerName="payload-processing"
Sep 14 03:17:55.027038 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:55.027040    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c55cf6b4-ff06-406e-b64b-548320256283" containerName="payload-processing"
Sep 14 03:17:55.027307 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:55.027104    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c017b1f-7231-4672-8106-062df68418a7" containerName="revoke-keys"
Sep 14 03:17:55.027307 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:55.027114    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c017b1f-7231-4672-8106-062df68418a7" containerName="revoke-keys"
Sep 14 03:17:55.027415 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:55.027335    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c55cf6b4-ff06-406e-b64b-548320256283" containerName="payload-processing"
Sep 14 03:17:55.027415 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:55.027350    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c017b1f-7231-4672-8106-062df68418a7" containerName="revoke-keys"
Sep 14 03:17:55.033006 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:55.032708    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9"
Sep 14 03:17:55.035735 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:55.035712    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:17:55.038464 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:55.038435    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9"]
Sep 14 03:17:55.152773 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:55.152745    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/000112f3-478a-48b0-b0f1-f294faec72b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9\" (UID: \"000112f3-478a-48b0-b0f1-f294faec72b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9"
Sep 14 03:17:55.253827 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:55.253790    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/000112f3-478a-48b0-b0f1-f294faec72b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9\" (UID: \"000112f3-478a-48b0-b0f1-f294faec72b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9"
Sep 14 03:17:55.254412 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:55.254393    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/000112f3-478a-48b0-b0f1-f294faec72b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9\" (UID: \"000112f3-478a-48b0-b0f1-f294faec72b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9"
Sep 14 03:17:55.346213 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:55.346117    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9"
Sep 14 03:17:55.621219 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:55.621161    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9"]
Sep 14 03:17:55.622905 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:17:55.622881    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod000112f3_478a_48b0_b0f1_f294faec72b6.slice/crio-e6faaea500cac49ad6f075e63dd736626ca52ddd9b44d97fad932eae8437403d WatchSource:0}: Error finding container e6faaea500cac49ad6f075e63dd736626ca52ddd9b44d97fad932eae8437403d: Status 404 returned error can't find the container with id e6faaea500cac49ad6f075e63dd736626ca52ddd9b44d97fad932eae8437403d
Sep 14 03:17:55.657383 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:55.657342    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9" event={"ID":"000112f3-478a-48b0-b0f1-f294faec72b6","Type":"ContainerStarted","Data":"e6faaea500cac49ad6f075e63dd736626ca52ddd9b44d97fad932eae8437403d"}
Sep 14 03:17:56.664110 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:56.664076    2579 generic.go:356] "Generic (PLEG): container finished" podID="000112f3-478a-48b0-b0f1-f294faec72b6" containerID="f91532f21a5cb546279c9503b766e2692d16c9bcb0b32b8ebfc4165fcbf6e1ac" exitCode=0
Sep 14 03:17:56.664513 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:56.664164    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9" event={"ID":"000112f3-478a-48b0-b0f1-f294faec72b6","Type":"ContainerDied","Data":"f91532f21a5cb546279c9503b766e2692d16c9bcb0b32b8ebfc4165fcbf6e1ac"}
Sep 14 03:17:57.803378 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:57.803354    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9"
Sep 14 03:17:57.874327 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:57.874300    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/000112f3-478a-48b0-b0f1-f294faec72b6-maas-api-service-ca\") pod \"000112f3-478a-48b0-b0f1-f294faec72b6\" (UID: \"000112f3-478a-48b0-b0f1-f294faec72b6\") "
Sep 14 03:17:57.874628 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:57.874603    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/000112f3-478a-48b0-b0f1-f294faec72b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "000112f3-478a-48b0-b0f1-f294faec72b6" (UID: "000112f3-478a-48b0-b0f1-f294faec72b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:57.975825 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:57.975760    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/000112f3-478a-48b0-b0f1-f294faec72b6-maas-api-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:58.676557 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:58.676529    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9"
Sep 14 03:17:58.676735 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:58.676562    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9" event={"ID":"000112f3-478a-48b0-b0f1-f294faec72b6","Type":"ContainerDied","Data":"e6faaea500cac49ad6f075e63dd736626ca52ddd9b44d97fad932eae8437403d"}
Sep 14 03:17:58.676735 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:58.676610    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6faaea500cac49ad6f075e63dd736626ca52ddd9b44d97fad932eae8437403d"
Sep 14 03:17:59.928274 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.928168    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:17:59.992343 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.992315    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istiod-ca-cert\") pod \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") "
Sep 14 03:17:59.992488 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.992370    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-workload-certs\") pod \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") "
Sep 14 03:17:59.992488 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.992386    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-token\") pod \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") "
Sep 14 03:17:59.992488 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.992461    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dw87\" (UniqueName: \"kubernetes.io/projected/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-kube-api-access-7dw87\") pod \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") "
Sep 14 03:17:59.992488 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.992486    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-data\") pod \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") "
Sep 14 03:17:59.992683 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.992520    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-envoy\") pod \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") "
Sep 14 03:17:59.992683 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.992554    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-workload-socket\") pod \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") "
Sep 14 03:17:59.992683 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.992616    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-podinfo\") pod \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") "
Sep 14 03:17:59.992683 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.992632    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" (UID: "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:59.992683 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.992648    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-credential-socket\") pod \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\" (UID: \"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb\") "
Sep 14 03:17:59.992683 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.992667    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" (UID: "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:17:59.992984 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.992752    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-data" (OuterVolumeSpecName: "istio-data") pod "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" (UID: "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:59.992984 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.992822    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" (UID: "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:59.992984 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.992911    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" (UID: "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:59.993142 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.993006    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:59.993142 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.993018    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:59.993142 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.993027    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:59.993142 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.993035    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:59.993142 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.993048    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:17:59.995030 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.995002    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" (UID: "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:17:59.995138 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.995036    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" (UID: "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:17:59.995138 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.995054    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-token" (OuterVolumeSpecName: "istio-token") pod "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" (UID: "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:17:59.995138 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:17:59.995094    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-kube-api-access-7dw87" (OuterVolumeSpecName: "kube-api-access-7dw87") pod "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" (UID: "bd8e1b0e-92cf-4e3c-9952-d9248fc041eb"). InnerVolumeSpecName "kube-api-access-7dw87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:18:00.094095 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:00.094071    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dw87\" (UniqueName: \"kubernetes.io/projected/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-kube-api-access-7dw87\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:00.094095 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:00.094097    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:00.094331 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:00.094110    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:00.094331 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:00.094122    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:00.697994 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:00.697955    2579 generic.go:356] "Generic (PLEG): container finished" podID="bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" containerID="c9ad893fe4d4d8cfe7de824e9125d0bb527b666501e661df49c5854004ec9539" exitCode=0
Sep 14 03:18:00.698227 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:00.698029    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"
Sep 14 03:18:00.698227 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:00.698037    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9" event={"ID":"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb","Type":"ContainerDied","Data":"c9ad893fe4d4d8cfe7de824e9125d0bb527b666501e661df49c5854004ec9539"}
Sep 14 03:18:00.698227 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:00.698129    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9" event={"ID":"bd8e1b0e-92cf-4e3c-9952-d9248fc041eb","Type":"ContainerDied","Data":"04fc05068df8559cc912bca343c408a379395ad6c93857cb66a523fa7da8537a"}
Sep 14 03:18:00.698227 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:00.698160    2579 scope.go:117] "RemoveContainer" containerID="c9ad893fe4d4d8cfe7de824e9125d0bb527b666501e661df49c5854004ec9539"
Sep 14 03:18:00.708748 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:00.708731    2579 scope.go:117] "RemoveContainer" containerID="15d648cb05738e9010c7f72cbaea1ce3528ea3651c228ea53dcf2f37f81ba47b"
Sep 14 03:18:00.716819 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:00.716806    2579 scope.go:117] "RemoveContainer" containerID="c9ad893fe4d4d8cfe7de824e9125d0bb527b666501e661df49c5854004ec9539"
Sep 14 03:18:00.717060 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:18:00.717043    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9ad893fe4d4d8cfe7de824e9125d0bb527b666501e661df49c5854004ec9539\": container with ID starting with c9ad893fe4d4d8cfe7de824e9125d0bb527b666501e661df49c5854004ec9539 not found: ID does not exist" containerID="c9ad893fe4d4d8cfe7de824e9125d0bb527b666501e661df49c5854004ec9539"
Sep 14 03:18:00.717105 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:00.717068    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9ad893fe4d4d8cfe7de824e9125d0bb527b666501e661df49c5854004ec9539"} err="failed to get container status \"c9ad893fe4d4d8cfe7de824e9125d0bb527b666501e661df49c5854004ec9539\": rpc error: code = NotFound desc = could not find container \"c9ad893fe4d4d8cfe7de824e9125d0bb527b666501e661df49c5854004ec9539\": container with ID starting with c9ad893fe4d4d8cfe7de824e9125d0bb527b666501e661df49c5854004ec9539 not found: ID does not exist"
Sep 14 03:18:00.717105 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:00.717081    2579 scope.go:117] "RemoveContainer" containerID="15d648cb05738e9010c7f72cbaea1ce3528ea3651c228ea53dcf2f37f81ba47b"
Sep 14 03:18:00.717665 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:18:00.717307    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15d648cb05738e9010c7f72cbaea1ce3528ea3651c228ea53dcf2f37f81ba47b\": container with ID starting with 15d648cb05738e9010c7f72cbaea1ce3528ea3651c228ea53dcf2f37f81ba47b not found: ID does not exist" containerID="15d648cb05738e9010c7f72cbaea1ce3528ea3651c228ea53dcf2f37f81ba47b"
Sep 14 03:18:00.717665 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:00.717324    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15d648cb05738e9010c7f72cbaea1ce3528ea3651c228ea53dcf2f37f81ba47b"} err="failed to get container status \"15d648cb05738e9010c7f72cbaea1ce3528ea3651c228ea53dcf2f37f81ba47b\": rpc error: code = NotFound desc = could not find container \"15d648cb05738e9010c7f72cbaea1ce3528ea3651c228ea53dcf2f37f81ba47b\": container with ID starting with 15d648cb05738e9010c7f72cbaea1ce3528ea3651c228ea53dcf2f37f81ba47b not found: ID does not exist"
Sep 14 03:18:00.721467 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:00.721447    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"]
Sep 14 03:18:00.725432 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:00.725415    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ce5b9b-openshift-default-6cf7cf5bbb-mdmw9"]
Sep 14 03:18:01.743669 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:01.743634    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" path="/var/lib/kubelet/pods/bd8e1b0e-92cf-4e3c-9952-d9248fc041eb/volumes"
Sep 14 03:18:03.881317 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:03.881281    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"]
Sep 14 03:18:03.881864 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:03.881537    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb" podUID="d70a57b9-05c5-47ac-9ad7-691033cc0edf" containerName="payload-pre-processing" containerID="cri-o://5c3972e0ba3410c26c96f4d1f0ec6a0667ad53206c0924492cb8cd550329adc4" gracePeriod=30
Sep 14 03:18:04.104896 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.104854    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn"]
Sep 14 03:18:04.107063 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.107038    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-9c9f1e-hr6qn"]
Sep 14 03:18:04.135548 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.135499    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"
Sep 14 03:18:04.233037 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.233008    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d70a57b9-05c5-47ac-9ad7-691033cc0edf-plugins-config-volume\") pod \"d70a57b9-05c5-47ac-9ad7-691033cc0edf\" (UID: \"d70a57b9-05c5-47ac-9ad7-691033cc0edf\") "
Sep 14 03:18:04.233220 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.233060    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8th7d\" (UniqueName: \"kubernetes.io/projected/d70a57b9-05c5-47ac-9ad7-691033cc0edf-kube-api-access-8th7d\") pod \"d70a57b9-05c5-47ac-9ad7-691033cc0edf\" (UID: \"d70a57b9-05c5-47ac-9ad7-691033cc0edf\") "
Sep 14 03:18:04.233415 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.233395    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d70a57b9-05c5-47ac-9ad7-691033cc0edf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d70a57b9-05c5-47ac-9ad7-691033cc0edf" (UID: "d70a57b9-05c5-47ac-9ad7-691033cc0edf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:18:04.235231 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.235209    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d70a57b9-05c5-47ac-9ad7-691033cc0edf-kube-api-access-8th7d" (OuterVolumeSpecName: "kube-api-access-8th7d") pod "d70a57b9-05c5-47ac-9ad7-691033cc0edf" (UID: "d70a57b9-05c5-47ac-9ad7-691033cc0edf"). InnerVolumeSpecName "kube-api-access-8th7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:18:04.334006 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.333974    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d70a57b9-05c5-47ac-9ad7-691033cc0edf-plugins-config-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:04.334006 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.333999    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8th7d\" (UniqueName: \"kubernetes.io/projected/d70a57b9-05c5-47ac-9ad7-691033cc0edf-kube-api-access-8th7d\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:04.366321 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.366287    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx" podUID="7d96d89d-13aa-4257-936b-eba41890eeb2" containerName="main" containerID="cri-o://e306a381bfac55ade345e09cd0b5646772a8d5fcdcb907af9ab60fdedd57c63f" gracePeriod=45
Sep 14 03:18:04.618212 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.617941    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:18:04.722945 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.722913    2579 generic.go:356] "Generic (PLEG): container finished" podID="d70a57b9-05c5-47ac-9ad7-691033cc0edf" containerID="5c3972e0ba3410c26c96f4d1f0ec6a0667ad53206c0924492cb8cd550329adc4" exitCode=0
Sep 14 03:18:04.723125 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.722967    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"
Sep 14 03:18:04.723125 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.722989    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb" event={"ID":"d70a57b9-05c5-47ac-9ad7-691033cc0edf","Type":"ContainerDied","Data":"5c3972e0ba3410c26c96f4d1f0ec6a0667ad53206c0924492cb8cd550329adc4"}
Sep 14 03:18:04.723125 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.723073    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb" event={"ID":"d70a57b9-05c5-47ac-9ad7-691033cc0edf","Type":"ContainerDied","Data":"62669acc8def4c440299594e6bec094d618a1964eb9499bf12d38c3eb2c8672d"}
Sep 14 03:18:04.723125 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.723104    2579 scope.go:117] "RemoveContainer" containerID="5c3972e0ba3410c26c96f4d1f0ec6a0667ad53206c0924492cb8cd550329adc4"
Sep 14 03:18:04.724593 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.724571    2579 generic.go:356] "Generic (PLEG): container finished" podID="7d96d89d-13aa-4257-936b-eba41890eeb2" containerID="e306a381bfac55ade345e09cd0b5646772a8d5fcdcb907af9ab60fdedd57c63f" exitCode=0
Sep 14 03:18:04.724729 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.724624    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"
Sep 14 03:18:04.724729 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.724648    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx" event={"ID":"7d96d89d-13aa-4257-936b-eba41890eeb2","Type":"ContainerDied","Data":"e306a381bfac55ade345e09cd0b5646772a8d5fcdcb907af9ab60fdedd57c63f"}
Sep 14 03:18:04.724729 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.724681    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx" event={"ID":"7d96d89d-13aa-4257-936b-eba41890eeb2","Type":"ContainerDied","Data":"b11ca64040281ea1b393222db192e48e7e9afe9b0c170dea430cf0f01890396b"}
Sep 14 03:18:04.733201 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.733130    2579 scope.go:117] "RemoveContainer" containerID="5c3972e0ba3410c26c96f4d1f0ec6a0667ad53206c0924492cb8cd550329adc4"
Sep 14 03:18:04.733461 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:18:04.733443    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c3972e0ba3410c26c96f4d1f0ec6a0667ad53206c0924492cb8cd550329adc4\": container with ID starting with 5c3972e0ba3410c26c96f4d1f0ec6a0667ad53206c0924492cb8cd550329adc4 not found: ID does not exist" containerID="5c3972e0ba3410c26c96f4d1f0ec6a0667ad53206c0924492cb8cd550329adc4"
Sep 14 03:18:04.733504 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.733469    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c3972e0ba3410c26c96f4d1f0ec6a0667ad53206c0924492cb8cd550329adc4"} err="failed to get container status \"5c3972e0ba3410c26c96f4d1f0ec6a0667ad53206c0924492cb8cd550329adc4\": rpc error: code = NotFound desc = could not find container \"5c3972e0ba3410c26c96f4d1f0ec6a0667ad53206c0924492cb8cd550329adc4\": container with ID starting with 5c3972e0ba3410c26c96f4d1f0ec6a0667ad53206c0924492cb8cd550329adc4 not found: ID does not exist"
Sep 14 03:18:04.733504 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.733485    2579 scope.go:117] "RemoveContainer" containerID="e306a381bfac55ade345e09cd0b5646772a8d5fcdcb907af9ab60fdedd57c63f"
Sep 14 03:18:04.737707 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.737688    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d96d89d-13aa-4257-936b-eba41890eeb2-tls-certs\") pod \"7d96d89d-13aa-4257-936b-eba41890eeb2\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") "
Sep 14 03:18:04.737803 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.737717    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-model-cache\") pod \"7d96d89d-13aa-4257-936b-eba41890eeb2\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") "
Sep 14 03:18:04.737803 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.737748    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfjbr\" (UniqueName: \"kubernetes.io/projected/7d96d89d-13aa-4257-936b-eba41890eeb2-kube-api-access-vfjbr\") pod \"7d96d89d-13aa-4257-936b-eba41890eeb2\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") "
Sep 14 03:18:04.737971 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.737832    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-home\") pod \"7d96d89d-13aa-4257-936b-eba41890eeb2\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") "
Sep 14 03:18:04.737971 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.737928    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-dshm\") pod \"7d96d89d-13aa-4257-936b-eba41890eeb2\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") "
Sep 14 03:18:04.738081 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.737974    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-tmp-dir\") pod \"7d96d89d-13aa-4257-936b-eba41890eeb2\" (UID: \"7d96d89d-13aa-4257-936b-eba41890eeb2\") "
Sep 14 03:18:04.738081 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.738009    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-model-cache" (OuterVolumeSpecName: "model-cache") pod "7d96d89d-13aa-4257-936b-eba41890eeb2" (UID: "7d96d89d-13aa-4257-936b-eba41890eeb2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:18:04.738271 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.738242    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-home" (OuterVolumeSpecName: "home") pod "7d96d89d-13aa-4257-936b-eba41890eeb2" (UID: "7d96d89d-13aa-4257-936b-eba41890eeb2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:18:04.738344 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.738260    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-model-cache\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:04.738568 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.738362    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7d96d89d-13aa-4257-936b-eba41890eeb2" (UID: "7d96d89d-13aa-4257-936b-eba41890eeb2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:18:04.740030 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.740004    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d96d89d-13aa-4257-936b-eba41890eeb2-kube-api-access-vfjbr" (OuterVolumeSpecName: "kube-api-access-vfjbr") pod "7d96d89d-13aa-4257-936b-eba41890eeb2" (UID: "7d96d89d-13aa-4257-936b-eba41890eeb2"). InnerVolumeSpecName "kube-api-access-vfjbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:18:04.740030 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.740020    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-dshm" (OuterVolumeSpecName: "dshm") pod "7d96d89d-13aa-4257-936b-eba41890eeb2" (UID: "7d96d89d-13aa-4257-936b-eba41890eeb2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:18:04.740144 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.740024    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d96d89d-13aa-4257-936b-eba41890eeb2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7d96d89d-13aa-4257-936b-eba41890eeb2" (UID: "7d96d89d-13aa-4257-936b-eba41890eeb2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:18:04.741949 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.741931    2579 scope.go:117] "RemoveContainer" containerID="e306a381bfac55ade345e09cd0b5646772a8d5fcdcb907af9ab60fdedd57c63f"
Sep 14 03:18:04.742230 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:18:04.742211    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e306a381bfac55ade345e09cd0b5646772a8d5fcdcb907af9ab60fdedd57c63f\": container with ID starting with e306a381bfac55ade345e09cd0b5646772a8d5fcdcb907af9ab60fdedd57c63f not found: ID does not exist" containerID="e306a381bfac55ade345e09cd0b5646772a8d5fcdcb907af9ab60fdedd57c63f"
Sep 14 03:18:04.742279 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.742239    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e306a381bfac55ade345e09cd0b5646772a8d5fcdcb907af9ab60fdedd57c63f"} err="failed to get container status \"e306a381bfac55ade345e09cd0b5646772a8d5fcdcb907af9ab60fdedd57c63f\": rpc error: code = NotFound desc = could not find container \"e306a381bfac55ade345e09cd0b5646772a8d5fcdcb907af9ab60fdedd57c63f\": container with ID starting with e306a381bfac55ade345e09cd0b5646772a8d5fcdcb907af9ab60fdedd57c63f not found: ID does not exist"
Sep 14 03:18:04.746858 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.746839    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"]
Sep 14 03:18:04.752376 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.752358    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-9c9f1e-5585495d58-nqmhb"]
Sep 14 03:18:04.839660 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.839630    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-dshm\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:04.839660 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.839658    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-tmp-dir\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:04.839845 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.839670    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d96d89d-13aa-4257-936b-eba41890eeb2-tls-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:04.839845 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.839686    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfjbr\" (UniqueName: \"kubernetes.io/projected/7d96d89d-13aa-4257-936b-eba41890eeb2-kube-api-access-vfjbr\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:04.839845 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:04.839699    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d96d89d-13aa-4257-936b-eba41890eeb2-home\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:05.046132 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:05.046099    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"]
Sep 14 03:18:05.058671 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:05.058651    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-9c9f1e/premium-w4-9c9f1e-kserve-557d986567-2jdhx"]
Sep 14 03:18:05.742879 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:05.742847    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d96d89d-13aa-4257-936b-eba41890eeb2" path="/var/lib/kubelet/pods/7d96d89d-13aa-4257-936b-eba41890eeb2/volumes"
Sep 14 03:18:05.743228 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:05.743215    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c017b1f-7231-4672-8106-062df68418a7" path="/var/lib/kubelet/pods/8c017b1f-7231-4672-8106-062df68418a7/volumes"
Sep 14 03:18:05.743529 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:05.743518    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d70a57b9-05c5-47ac-9ad7-691033cc0edf" path="/var/lib/kubelet/pods/d70a57b9-05c5-47ac-9ad7-691033cc0edf/volumes"
Sep 14 03:18:07.535288 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:07.535259    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"]
Sep 14 03:18:07.743655 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:07.743602    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz" podUID="54182426-00ba-4567-9347-054e89dd65dc" containerName="payload-pre-processing" containerID="cri-o://333f227f32b83e044752fb019bcadb970b0527b9116876b3a72184b4a5df7783" gracePeriod=30
Sep 14 03:18:07.946065 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:07.944887    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9"]
Sep 14 03:18:07.951461 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:07.951432    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-17d8b8-25sx9"]
Sep 14 03:18:08.056800 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:08.056778    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"
Sep 14 03:18:08.170418 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:08.170350    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54182426-00ba-4567-9347-054e89dd65dc-plugins-config-volume\") pod \"54182426-00ba-4567-9347-054e89dd65dc\" (UID: \"54182426-00ba-4567-9347-054e89dd65dc\") "
Sep 14 03:18:08.170557 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:08.170420    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djvcn\" (UniqueName: \"kubernetes.io/projected/54182426-00ba-4567-9347-054e89dd65dc-kube-api-access-djvcn\") pod \"54182426-00ba-4567-9347-054e89dd65dc\" (UID: \"54182426-00ba-4567-9347-054e89dd65dc\") "
Sep 14 03:18:08.170719 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:08.170694    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54182426-00ba-4567-9347-054e89dd65dc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "54182426-00ba-4567-9347-054e89dd65dc" (UID: "54182426-00ba-4567-9347-054e89dd65dc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:18:08.172677 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:08.172659    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54182426-00ba-4567-9347-054e89dd65dc-kube-api-access-djvcn" (OuterVolumeSpecName: "kube-api-access-djvcn") pod "54182426-00ba-4567-9347-054e89dd65dc" (UID: "54182426-00ba-4567-9347-054e89dd65dc"). InnerVolumeSpecName "kube-api-access-djvcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:18:08.271468 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:08.271445    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54182426-00ba-4567-9347-054e89dd65dc-plugins-config-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:08.271468 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:08.271466    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djvcn\" (UniqueName: \"kubernetes.io/projected/54182426-00ba-4567-9347-054e89dd65dc-kube-api-access-djvcn\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:08.750197 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:08.750157    2579 generic.go:356] "Generic (PLEG): container finished" podID="54182426-00ba-4567-9347-054e89dd65dc" containerID="333f227f32b83e044752fb019bcadb970b0527b9116876b3a72184b4a5df7783" exitCode=0
Sep 14 03:18:08.750602 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:08.750214    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz" event={"ID":"54182426-00ba-4567-9347-054e89dd65dc","Type":"ContainerDied","Data":"333f227f32b83e044752fb019bcadb970b0527b9116876b3a72184b4a5df7783"}
Sep 14 03:18:08.750602 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:08.750265    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz" event={"ID":"54182426-00ba-4567-9347-054e89dd65dc","Type":"ContainerDied","Data":"efb1f56a51fb5db8b9d52d4f3f35fdc8dd4fb18bd136ee7e1cb6b6a941f3a515"}
Sep 14 03:18:08.750602 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:08.750286    2579 scope.go:117] "RemoveContainer" containerID="333f227f32b83e044752fb019bcadb970b0527b9116876b3a72184b4a5df7783"
Sep 14 03:18:08.750602 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:08.750222    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"
Sep 14 03:18:08.759448 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:08.759432    2579 scope.go:117] "RemoveContainer" containerID="333f227f32b83e044752fb019bcadb970b0527b9116876b3a72184b4a5df7783"
Sep 14 03:18:08.759693 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:18:08.759672    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"333f227f32b83e044752fb019bcadb970b0527b9116876b3a72184b4a5df7783\": container with ID starting with 333f227f32b83e044752fb019bcadb970b0527b9116876b3a72184b4a5df7783 not found: ID does not exist" containerID="333f227f32b83e044752fb019bcadb970b0527b9116876b3a72184b4a5df7783"
Sep 14 03:18:08.759755 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:08.759704    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"333f227f32b83e044752fb019bcadb970b0527b9116876b3a72184b4a5df7783"} err="failed to get container status \"333f227f32b83e044752fb019bcadb970b0527b9116876b3a72184b4a5df7783\": rpc error: code = NotFound desc = could not find container \"333f227f32b83e044752fb019bcadb970b0527b9116876b3a72184b4a5df7783\": container with ID starting with 333f227f32b83e044752fb019bcadb970b0527b9116876b3a72184b4a5df7783 not found: ID does not exist"
Sep 14 03:18:08.773075 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:08.773052    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"]
Sep 14 03:18:08.776357 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:08.776342    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-17d8b8-6cd4f6d65d-dmzrz"]
Sep 14 03:18:09.750416 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:09.750386    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="000112f3-478a-48b0-b0f1-f294faec72b6" path="/var/lib/kubelet/pods/000112f3-478a-48b0-b0f1-f294faec72b6/volumes"
Sep 14 03:18:09.750815 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:09.750754    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54182426-00ba-4567-9347-054e89dd65dc" path="/var/lib/kubelet/pods/54182426-00ba-4567-9347-054e89dd65dc/volumes"
Sep 14 03:18:14.698105 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:14.698062    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"]
Sep 14 03:18:14.698558 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:14.698352    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk" podUID="b13dad7c-b629-4395-b830-005b15777742" containerName="istio-proxy" containerID="cri-o://86a2339c61db33642434518f6afb89247e8e2b7e83b32fd589ae4f60e41d9138" gracePeriod=30
Sep 14 03:18:14.975981 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:14.975906    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 03:18:14.975981 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:14.975959    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk" podUID="b13dad7c-b629-4395-b830-005b15777742" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 03:18:17.975821 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:17.975788    2579 scope.go:117] "RemoveContainer" containerID="565c03ea30b7fe9a1b6acc8b8266a43d106b44478e1816dae96b30d74fdeab39"
Sep 14 03:18:19.815573 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:19.815544    2579 generic.go:356] "Generic (PLEG): container finished" podID="b13dad7c-b629-4395-b830-005b15777742" containerID="86a2339c61db33642434518f6afb89247e8e2b7e83b32fd589ae4f60e41d9138" exitCode=0
Sep 14 03:18:19.815910 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:19.815601    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk" event={"ID":"b13dad7c-b629-4395-b830-005b15777742","Type":"ContainerDied","Data":"86a2339c61db33642434518f6afb89247e8e2b7e83b32fd589ae4f60e41d9138"}
Sep 14 03:18:19.815910 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:19.815657    2579 scope.go:117] "RemoveContainer" containerID="f161e3951af4f8e85765737e2c9be145cda43a36bfb5e9a97e9ee496f4f8545d"
Sep 14 03:18:19.950372 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:19.950347    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:18:20.077943 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.077919    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-istio-envoy\") pod \"b13dad7c-b629-4395-b830-005b15777742\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") "
Sep 14 03:18:20.078130 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.077953    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b13dad7c-b629-4395-b830-005b15777742-istiod-ca-cert\") pod \"b13dad7c-b629-4395-b830-005b15777742\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") "
Sep 14 03:18:20.078130 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.077982    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-workload-socket\") pod \"b13dad7c-b629-4395-b830-005b15777742\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") "
Sep 14 03:18:20.078130 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.078007    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b13dad7c-b629-4395-b830-005b15777742-istio-token\") pod \"b13dad7c-b629-4395-b830-005b15777742\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") "
Sep 14 03:18:20.078130 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.078025    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b13dad7c-b629-4395-b830-005b15777742-istio-podinfo\") pod \"b13dad7c-b629-4395-b830-005b15777742\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") "
Sep 14 03:18:20.078130 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.078068    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-istio-data\") pod \"b13dad7c-b629-4395-b830-005b15777742\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") "
Sep 14 03:18:20.078130 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.078087    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smbnc\" (UniqueName: \"kubernetes.io/projected/b13dad7c-b629-4395-b830-005b15777742-kube-api-access-smbnc\") pod \"b13dad7c-b629-4395-b830-005b15777742\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") "
Sep 14 03:18:20.078488 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.078140    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-workload-certs\") pod \"b13dad7c-b629-4395-b830-005b15777742\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") "
Sep 14 03:18:20.078488 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.078213    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-credential-socket\") pod \"b13dad7c-b629-4395-b830-005b15777742\" (UID: \"b13dad7c-b629-4395-b830-005b15777742\") "
Sep 14 03:18:20.078488 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.078267    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b13dad7c-b629-4395-b830-005b15777742" (UID: "b13dad7c-b629-4395-b830-005b15777742"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:18:20.078488 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.078417    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-istio-data" (OuterVolumeSpecName: "istio-data") pod "b13dad7c-b629-4395-b830-005b15777742" (UID: "b13dad7c-b629-4395-b830-005b15777742"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:18:20.078488 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.078468    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b13dad7c-b629-4395-b830-005b15777742-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b13dad7c-b629-4395-b830-005b15777742" (UID: "b13dad7c-b629-4395-b830-005b15777742"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:18:20.078734 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.078545    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b13dad7c-b629-4395-b830-005b15777742" (UID: "b13dad7c-b629-4395-b830-005b15777742"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:18:20.078734 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.078611    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b13dad7c-b629-4395-b830-005b15777742" (UID: "b13dad7c-b629-4395-b830-005b15777742"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:18:20.078734 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.078635    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b13dad7c-b629-4395-b830-005b15777742-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:20.078734 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.078653    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:20.078734 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.078662    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:20.080402 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.080376    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b13dad7c-b629-4395-b830-005b15777742-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b13dad7c-b629-4395-b830-005b15777742" (UID: "b13dad7c-b629-4395-b830-005b15777742"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:18:20.080489 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.080380    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b13dad7c-b629-4395-b830-005b15777742" (UID: "b13dad7c-b629-4395-b830-005b15777742"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:18:20.080814 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.080793    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b13dad7c-b629-4395-b830-005b15777742-istio-token" (OuterVolumeSpecName: "istio-token") pod "b13dad7c-b629-4395-b830-005b15777742" (UID: "b13dad7c-b629-4395-b830-005b15777742"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:18:20.080912 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.080819    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b13dad7c-b629-4395-b830-005b15777742-kube-api-access-smbnc" (OuterVolumeSpecName: "kube-api-access-smbnc") pod "b13dad7c-b629-4395-b830-005b15777742" (UID: "b13dad7c-b629-4395-b830-005b15777742"). InnerVolumeSpecName "kube-api-access-smbnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:18:20.179050 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.179021    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smbnc\" (UniqueName: \"kubernetes.io/projected/b13dad7c-b629-4395-b830-005b15777742-kube-api-access-smbnc\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:20.179050 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.179044    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:20.179050 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.179052    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:20.179290 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.179061    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b13dad7c-b629-4395-b830-005b15777742-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:20.179290 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.179068    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b13dad7c-b629-4395-b830-005b15777742-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:20.179290 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.179076    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b13dad7c-b629-4395-b830-005b15777742-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:18:20.822298 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.822267    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"
Sep 14 03:18:20.822298 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.822275    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk" event={"ID":"b13dad7c-b629-4395-b830-005b15777742","Type":"ContainerDied","Data":"45bfaf90d6f6ded9d880d934394331ced9a4ee9bb28fd9996a5b2424be07dfd2"}
Sep 14 03:18:20.822749 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.822334    2579 scope.go:117] "RemoveContainer" containerID="86a2339c61db33642434518f6afb89247e8e2b7e83b32fd589ae4f60e41d9138"
Sep 14 03:18:20.843731 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.843705    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"]
Sep 14 03:18:20.848107 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:20.848087    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-17d8b8-openshift-default-6f74899868-d29rk"]
Sep 14 03:18:21.743729 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:18:21.743698    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b13dad7c-b629-4395-b830-005b15777742" path="/var/lib/kubelet/pods/b13dad7c-b629-4395-b830-005b15777742/volumes"
Sep 14 03:20:18.036350 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:20:18.036315    2579 scope.go:117] "RemoveContainer" containerID="31b7a50275d151f8825e7e71305f00822ae081032bd4e777d73724f431d626e5"
Sep 14 03:21:18.054965 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:18.054932    2579 scope.go:117] "RemoveContainer" containerID="e48ffc00d5ec12eea834dc8515313f9b96d67cf6ecf903641fddc7eb97c31d06"
Sep 14 03:21:18.063138 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:18.063112    2579 scope.go:117] "RemoveContainer" containerID="4bb5c38bed0f5d91a4e350f2cf9b3845dbf83786f2f1d6e10ad846934a040025"
Sep 14 03:21:31.191472 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191437    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"]
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191778    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b13dad7c-b629-4395-b830-005b15777742" containerName="istio-proxy"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191788    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b13dad7c-b629-4395-b830-005b15777742" containerName="istio-proxy"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191797    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54182426-00ba-4567-9347-054e89dd65dc" containerName="payload-pre-processing"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191802    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="54182426-00ba-4567-9347-054e89dd65dc" containerName="payload-pre-processing"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191809    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b13dad7c-b629-4395-b830-005b15777742" containerName="istio-proxy"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191814    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b13dad7c-b629-4395-b830-005b15777742" containerName="istio-proxy"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191823    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="000112f3-478a-48b0-b0f1-f294faec72b6" containerName="revoke-keys"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191829    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="000112f3-478a-48b0-b0f1-f294faec72b6" containerName="revoke-keys"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191839    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d70a57b9-05c5-47ac-9ad7-691033cc0edf" containerName="payload-pre-processing"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191844    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d70a57b9-05c5-47ac-9ad7-691033cc0edf" containerName="payload-pre-processing"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191852    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" containerName="istio-proxy"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191857    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" containerName="istio-proxy"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191866    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" containerName="istio-proxy"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191871    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" containerName="istio-proxy"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191879    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d96d89d-13aa-4257-936b-eba41890eeb2" containerName="main"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191885    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d96d89d-13aa-4257-936b-eba41890eeb2" containerName="main"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191935    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d70a57b9-05c5-47ac-9ad7-691033cc0edf" containerName="payload-pre-processing"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191942    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d96d89d-13aa-4257-936b-eba41890eeb2" containerName="main"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191948    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" containerName="istio-proxy"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191953    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd8e1b0e-92cf-4e3c-9952-d9248fc041eb" containerName="istio-proxy"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191959    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b13dad7c-b629-4395-b830-005b15777742" containerName="istio-proxy"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191965    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="54182426-00ba-4567-9347-054e89dd65dc" containerName="payload-pre-processing"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191970    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="000112f3-478a-48b0-b0f1-f294faec72b6" containerName="revoke-keys"
Sep 14 03:21:31.194028 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.191976    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b13dad7c-b629-4395-b830-005b15777742" containerName="istio-proxy"
Sep 14 03:21:31.195320 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.194867    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"
Sep 14 03:21:31.198102 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.198080    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-d081a3\""
Sep 14 03:21:31.198652 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.198637    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-d081a3-dockercfg-46cdr\""
Sep 14 03:21:31.215862 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.215839    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"]
Sep 14 03:21:31.321134 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.321075    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvxpm\" (UniqueName: \"kubernetes.io/projected/9f17d7d4-7aba-47fd-80f8-5513f4645b98-kube-api-access-bvxpm\") pod \"payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c\" (UID: \"9f17d7d4-7aba-47fd-80f8-5513f4645b98\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"
Sep 14 03:21:31.321352 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.321207    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f17d7d4-7aba-47fd-80f8-5513f4645b98-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c\" (UID: \"9f17d7d4-7aba-47fd-80f8-5513f4645b98\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"
Sep 14 03:21:31.422142 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.422113    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvxpm\" (UniqueName: \"kubernetes.io/projected/9f17d7d4-7aba-47fd-80f8-5513f4645b98-kube-api-access-bvxpm\") pod \"payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c\" (UID: \"9f17d7d4-7aba-47fd-80f8-5513f4645b98\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"
Sep 14 03:21:31.422323 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.422155    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f17d7d4-7aba-47fd-80f8-5513f4645b98-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c\" (UID: \"9f17d7d4-7aba-47fd-80f8-5513f4645b98\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"
Sep 14 03:21:31.423063 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.423040    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f17d7d4-7aba-47fd-80f8-5513f4645b98-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c\" (UID: \"9f17d7d4-7aba-47fd-80f8-5513f4645b98\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"
Sep 14 03:21:31.442384 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.442326    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvxpm\" (UniqueName: \"kubernetes.io/projected/9f17d7d4-7aba-47fd-80f8-5513f4645b98-kube-api-access-bvxpm\") pod \"payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c\" (UID: \"9f17d7d4-7aba-47fd-80f8-5513f4645b98\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"
Sep 14 03:21:31.505447 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.505423    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"
Sep 14 03:21:31.775122 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.775101    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"]
Sep 14 03:21:31.777325 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:21:31.777297    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f17d7d4_7aba_47fd_80f8_5513f4645b98.slice/crio-990d65ad57204427ce51198fae247851a9c1c5c41650efc97071217d7a6fce5b WatchSource:0}: Error finding container 990d65ad57204427ce51198fae247851a9c1c5c41650efc97071217d7a6fce5b: Status 404 returned error can't find the container with id 990d65ad57204427ce51198fae247851a9c1c5c41650efc97071217d7a6fce5b
Sep 14 03:21:31.822338 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:31.822297    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c" event={"ID":"9f17d7d4-7aba-47fd-80f8-5513f4645b98","Type":"ContainerStarted","Data":"990d65ad57204427ce51198fae247851a9c1c5c41650efc97071217d7a6fce5b"}
Sep 14 03:21:32.829837 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:32.829797    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c" event={"ID":"9f17d7d4-7aba-47fd-80f8-5513f4645b98","Type":"ContainerStarted","Data":"d8c9bf990bbcc2eb90d1d12b01be2df30f060bde86c49534cf135fc8464cafe9"}
Sep 14 03:21:32.830253 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:32.829874    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"
Sep 14 03:21:32.849939 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:32.849889    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c" podStartSLOduration=1.849874592 podStartE2EDuration="1.849874592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:21:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:21:32.849283191 +0000 UTC m=+1755.635345115" watchObservedRunningTime="2026-09-14 03:21:32.849874592 +0000 UTC m=+1755.635936482"
Sep 14 03:21:43.837145 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:21:43.837116    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"
Sep 14 03:22:54.727208 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.727140    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"]
Sep 14 03:22:54.730655 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.730636    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.733873 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.733844    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-3b049b-openshift-default-dockercfg-q2ksb\""
Sep 14 03:22:54.748008 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.747985    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"]
Sep 14 03:22:54.842944 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.842912    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-workload-certs\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.843123 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.842965    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istiod-ca-cert\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.843123 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.843024    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lv7c\" (UniqueName: \"kubernetes.io/projected/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-kube-api-access-5lv7c\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.843123 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.843050    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-data\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.843123 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.843098    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-envoy\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.843444 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.843129    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-credential-socket\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.843444 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.843231    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-token\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.843444 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.843280    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-workload-socket\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.843444 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.843307    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-podinfo\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.944057 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.944017    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-workload-socket\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.944057 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.944054    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-podinfo\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.944328 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.944092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-workload-certs\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.944328 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.944116    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istiod-ca-cert\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.944328 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.944147    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lv7c\" (UniqueName: \"kubernetes.io/projected/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-kube-api-access-5lv7c\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.944328 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.944169    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-data\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.944328 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.944213    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-envoy\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.944328 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.944249    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-credential-socket\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.944328 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.944294    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-token\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.944732 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.944480    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-workload-socket\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.944732 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.944558    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-data\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.944732 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.944716    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-credential-socket\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.945052 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.944753    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-workload-certs\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.945052 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.944827    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istiod-ca-cert\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.946595 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.946575    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-envoy\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.946853 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.946837    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-podinfo\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.953412 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.953391    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lv7c\" (UniqueName: \"kubernetes.io/projected/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-kube-api-access-5lv7c\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:54.953625 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:54.953608    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-token\") pod \"e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") " pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:55.041543 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:55.041516    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:55.318928 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:55.318903    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"]
Sep 14 03:22:55.320810 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:22:55.320782    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc557a6a8_a252_47b5_bd0a_b37cf8ba1831.slice/crio-f2989a7f1a8e5abee37e2ce20e8b292c72ba6980ae752e415c3ed09073a927a5 WatchSource:0}: Error finding container f2989a7f1a8e5abee37e2ce20e8b292c72ba6980ae752e415c3ed09073a927a5: Status 404 returned error can't find the container with id f2989a7f1a8e5abee37e2ce20e8b292c72ba6980ae752e415c3ed09073a927a5
Sep 14 03:22:55.323226 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:55.323168    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:22:55.323312 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:55.323300    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:22:55.323361 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:55.323341    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:22:56.266959 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:56.266919    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt" event={"ID":"c557a6a8-a252-47b5-bd0a-b37cf8ba1831","Type":"ContainerStarted","Data":"e7f4660ca9e852c5f502ace7ba7a3ab038d57e927458ac663b7108a560f0ff8c"}
Sep 14 03:22:56.267341 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:56.266980    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt" event={"ID":"c557a6a8-a252-47b5-bd0a-b37cf8ba1831","Type":"ContainerStarted","Data":"f2989a7f1a8e5abee37e2ce20e8b292c72ba6980ae752e415c3ed09073a927a5"}
Sep 14 03:22:57.041799 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:57.041764    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:57.046631 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:57.046606    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:57.074651 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:57.074609    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt" podStartSLOduration=3.07459473 podStartE2EDuration="3.07459473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:22:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:22:56.296433626 +0000 UTC m=+1839.082495538" watchObservedRunningTime="2026-09-14 03:22:57.07459473 +0000 UTC m=+1839.860656643"
Sep 14 03:22:57.271842 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:57.271814    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:22:57.272672 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:22:57.272654    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:23:01.308394 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.308361    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"]
Sep 14 03:23:01.312374 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.312355    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:01.316162 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.316138    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-3b049b-serving-cert\""
Sep 14 03:23:01.322685 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.322660    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"]
Sep 14 03:23:01.406550 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.406511    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7395698b-326f-4632-9eab-fde9abb0f8f8-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd\" (UID: \"7395698b-326f-4632-9eab-fde9abb0f8f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:01.406550 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.406564    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtccw\" (UniqueName: \"kubernetes.io/projected/7395698b-326f-4632-9eab-fde9abb0f8f8-kube-api-access-jtccw\") pod \"maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd\" (UID: \"7395698b-326f-4632-9eab-fde9abb0f8f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:01.407117 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.406593    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7395698b-326f-4632-9eab-fde9abb0f8f8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd\" (UID: \"7395698b-326f-4632-9eab-fde9abb0f8f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:01.407117 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.406623    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7395698b-326f-4632-9eab-fde9abb0f8f8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd\" (UID: \"7395698b-326f-4632-9eab-fde9abb0f8f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:01.508094 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.508065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7395698b-326f-4632-9eab-fde9abb0f8f8-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd\" (UID: \"7395698b-326f-4632-9eab-fde9abb0f8f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:01.508094 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.508100    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtccw\" (UniqueName: \"kubernetes.io/projected/7395698b-326f-4632-9eab-fde9abb0f8f8-kube-api-access-jtccw\") pod \"maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd\" (UID: \"7395698b-326f-4632-9eab-fde9abb0f8f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:01.508362 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.508120    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7395698b-326f-4632-9eab-fde9abb0f8f8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd\" (UID: \"7395698b-326f-4632-9eab-fde9abb0f8f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:01.508362 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.508136    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7395698b-326f-4632-9eab-fde9abb0f8f8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd\" (UID: \"7395698b-326f-4632-9eab-fde9abb0f8f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:01.508854 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.508820    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7395698b-326f-4632-9eab-fde9abb0f8f8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd\" (UID: \"7395698b-326f-4632-9eab-fde9abb0f8f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:01.510836 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.510788    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7395698b-326f-4632-9eab-fde9abb0f8f8-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd\" (UID: \"7395698b-326f-4632-9eab-fde9abb0f8f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:01.510992 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.510966    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7395698b-326f-4632-9eab-fde9abb0f8f8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd\" (UID: \"7395698b-326f-4632-9eab-fde9abb0f8f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:01.517785 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.517763    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtccw\" (UniqueName: \"kubernetes.io/projected/7395698b-326f-4632-9eab-fde9abb0f8f8-kube-api-access-jtccw\") pod \"maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd\" (UID: \"7395698b-326f-4632-9eab-fde9abb0f8f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:01.624658 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.624585    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:01.907820 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.907791    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"]
Sep 14 03:23:01.911384 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:23:01.911358    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7395698b_326f_4632_9eab_fde9abb0f8f8.slice/crio-ddf69a0605ffa08eaee34582da94aaa4171411ca98ebeaf55d73eef3ab5510af WatchSource:0}: Error finding container ddf69a0605ffa08eaee34582da94aaa4171411ca98ebeaf55d73eef3ab5510af: Status 404 returned error can't find the container with id ddf69a0605ffa08eaee34582da94aaa4171411ca98ebeaf55d73eef3ab5510af
Sep 14 03:23:01.913328 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:01.913303    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 03:23:02.301165 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:02.301139    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd" event={"ID":"7395698b-326f-4632-9eab-fde9abb0f8f8","Type":"ContainerStarted","Data":"ddf69a0605ffa08eaee34582da94aaa4171411ca98ebeaf55d73eef3ab5510af"}
Sep 14 03:23:03.308275 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:03.308242    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd" event={"ID":"7395698b-326f-4632-9eab-fde9abb0f8f8","Type":"ContainerStarted","Data":"ffa8c11cfe276215bef10fec0e9aac1b909eeeb57cf3b1df7e750138be50ee1a"}
Sep 14 03:23:03.308647 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:03.308333    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:03.344072 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:03.344022    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd" podStartSLOduration=1.988297406 podStartE2EDuration="2.344005944s" totalImagesPullingTime="355.708538ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:23:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:23:03.343036507 +0000 UTC m=+1846.129098421" watchObservedRunningTime="2026-09-14 03:23:03.344005944 +0000 UTC m=+1846.130067857"
Sep 14 03:23:09.318447 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:09.318417    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:18.091762 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:18.091728    2579 scope.go:117] "RemoveContainer" containerID="6479acef64be0fc1d2b74a4ef397c5ff691fc9af4c0dea0a3d5045b40288bd53"
Sep 14 03:23:18.102664 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:18.102640    2579 scope.go:117] "RemoveContainer" containerID="c82d2390e576bddd8ba764d21a0882e13b80788ccb5bfe35c06196680bb9d17a"
Sep 14 03:23:18.111619 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:18.111598    2579 scope.go:117] "RemoveContainer" containerID="7c3be39f961d73e4eed663d1411c942fa94852f14497e7ed6830d6cb9bbad708"
Sep 14 03:23:18.393732 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:18.393662    2579 generic.go:356] "Generic (PLEG): container finished" podID="c557a6a8-a252-47b5-bd0a-b37cf8ba1831" containerID="e7f4660ca9e852c5f502ace7ba7a3ab038d57e927458ac663b7108a560f0ff8c" exitCode=0
Sep 14 03:23:18.393918 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:18.393741    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt" event={"ID":"c557a6a8-a252-47b5-bd0a-b37cf8ba1831","Type":"ContainerDied","Data":"e7f4660ca9e852c5f502ace7ba7a3ab038d57e927458ac663b7108a560f0ff8c"}
Sep 14 03:23:18.394212 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:18.394194    2579 scope.go:117] "RemoveContainer" containerID="e7f4660ca9e852c5f502ace7ba7a3ab038d57e927458ac663b7108a560f0ff8c"
Sep 14 03:23:18.395362 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:18.395332    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:23:18.395466 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:18.395404    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:23:18.395466 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:18.395440    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892132Ki","pods":"250"}
Sep 14 03:23:19.406388 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:19.406355    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt" event={"ID":"c557a6a8-a252-47b5-bd0a-b37cf8ba1831","Type":"ContainerStarted","Data":"8e031b562f9ce6a6550dba3f35f3b553f6026197617cd03d1afa58d36df76fb8"}
Sep 14 03:23:20.042738 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:20.042704    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:23:20.042971 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:20.042951    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 14 03:23:20.043019 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:20.042999    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt" podUID="c557a6a8-a252-47b5-bd0a-b37cf8ba1831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 14 03:23:21.042918 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:21.042882    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 14 03:23:21.043322 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:21.042954    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt" podUID="c557a6a8-a252-47b5-bd0a-b37cf8ba1831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 14 03:23:22.042570 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:22.042528    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 14 03:23:22.042747 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:22.042588    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt" podUID="c557a6a8-a252-47b5-bd0a-b37cf8ba1831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 14 03:23:23.042525 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:23.042487    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 14 03:23:23.042906 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:23.042555    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt" podUID="c557a6a8-a252-47b5-bd0a-b37cf8ba1831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 14 03:23:24.042397 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:24.042355    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 14 03:23:24.042571 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:24.042418    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt" podUID="c557a6a8-a252-47b5-bd0a-b37cf8ba1831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 14 03:23:25.046532 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:25.046503    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:23:25.047016 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:25.046879    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:23:25.047558 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:25.047538    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:23:40.472145 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:40.472052    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"]
Sep 14 03:23:40.472594 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:40.472317    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd" podUID="7395698b-326f-4632-9eab-fde9abb0f8f8" containerName="maas-api" containerID="cri-o://ffa8c11cfe276215bef10fec0e9aac1b909eeeb57cf3b1df7e750138be50ee1a" gracePeriod=30
Sep 14 03:23:40.727135 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:40.727081    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:40.778139 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:40.778112    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7395698b-326f-4632-9eab-fde9abb0f8f8-metrics-certs\") pod \"7395698b-326f-4632-9eab-fde9abb0f8f8\" (UID: \"7395698b-326f-4632-9eab-fde9abb0f8f8\") "
Sep 14 03:23:40.778290 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:40.778201    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtccw\" (UniqueName: \"kubernetes.io/projected/7395698b-326f-4632-9eab-fde9abb0f8f8-kube-api-access-jtccw\") pod \"7395698b-326f-4632-9eab-fde9abb0f8f8\" (UID: \"7395698b-326f-4632-9eab-fde9abb0f8f8\") "
Sep 14 03:23:40.778290 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:40.778231    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7395698b-326f-4632-9eab-fde9abb0f8f8-openshift-service-ca\") pod \"7395698b-326f-4632-9eab-fde9abb0f8f8\" (UID: \"7395698b-326f-4632-9eab-fde9abb0f8f8\") "
Sep 14 03:23:40.778392 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:40.778331    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7395698b-326f-4632-9eab-fde9abb0f8f8-maas-api-tls\") pod \"7395698b-326f-4632-9eab-fde9abb0f8f8\" (UID: \"7395698b-326f-4632-9eab-fde9abb0f8f8\") "
Sep 14 03:23:40.778585 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:40.778556    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7395698b-326f-4632-9eab-fde9abb0f8f8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7395698b-326f-4632-9eab-fde9abb0f8f8" (UID: "7395698b-326f-4632-9eab-fde9abb0f8f8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:23:40.778766 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:40.778745    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7395698b-326f-4632-9eab-fde9abb0f8f8-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:23:40.780496 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:40.780454    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7395698b-326f-4632-9eab-fde9abb0f8f8-kube-api-access-jtccw" (OuterVolumeSpecName: "kube-api-access-jtccw") pod "7395698b-326f-4632-9eab-fde9abb0f8f8" (UID: "7395698b-326f-4632-9eab-fde9abb0f8f8"). InnerVolumeSpecName "kube-api-access-jtccw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:23:40.780593 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:40.780462    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7395698b-326f-4632-9eab-fde9abb0f8f8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7395698b-326f-4632-9eab-fde9abb0f8f8" (UID: "7395698b-326f-4632-9eab-fde9abb0f8f8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:23:40.780839 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:40.780806    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7395698b-326f-4632-9eab-fde9abb0f8f8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7395698b-326f-4632-9eab-fde9abb0f8f8" (UID: "7395698b-326f-4632-9eab-fde9abb0f8f8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:23:40.880453 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:40.880338    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtccw\" (UniqueName: \"kubernetes.io/projected/7395698b-326f-4632-9eab-fde9abb0f8f8-kube-api-access-jtccw\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:23:40.880453 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:40.880379    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7395698b-326f-4632-9eab-fde9abb0f8f8-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:23:40.880453 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:40.880394    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7395698b-326f-4632-9eab-fde9abb0f8f8-metrics-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:23:41.523393 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:41.523357    2579 generic.go:356] "Generic (PLEG): container finished" podID="7395698b-326f-4632-9eab-fde9abb0f8f8" containerID="ffa8c11cfe276215bef10fec0e9aac1b909eeeb57cf3b1df7e750138be50ee1a" exitCode=0
Sep 14 03:23:41.523880 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:41.523410    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"
Sep 14 03:23:41.523880 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:41.523438    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd" event={"ID":"7395698b-326f-4632-9eab-fde9abb0f8f8","Type":"ContainerDied","Data":"ffa8c11cfe276215bef10fec0e9aac1b909eeeb57cf3b1df7e750138be50ee1a"}
Sep 14 03:23:41.523880 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:41.523491    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd" event={"ID":"7395698b-326f-4632-9eab-fde9abb0f8f8","Type":"ContainerDied","Data":"ddf69a0605ffa08eaee34582da94aaa4171411ca98ebeaf55d73eef3ab5510af"}
Sep 14 03:23:41.523880 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:41.523513    2579 scope.go:117] "RemoveContainer" containerID="ffa8c11cfe276215bef10fec0e9aac1b909eeeb57cf3b1df7e750138be50ee1a"
Sep 14 03:23:41.540163 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:41.540142    2579 scope.go:117] "RemoveContainer" containerID="ffa8c11cfe276215bef10fec0e9aac1b909eeeb57cf3b1df7e750138be50ee1a"
Sep 14 03:23:41.540433 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:23:41.540414    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffa8c11cfe276215bef10fec0e9aac1b909eeeb57cf3b1df7e750138be50ee1a\": container with ID starting with ffa8c11cfe276215bef10fec0e9aac1b909eeeb57cf3b1df7e750138be50ee1a not found: ID does not exist" containerID="ffa8c11cfe276215bef10fec0e9aac1b909eeeb57cf3b1df7e750138be50ee1a"
Sep 14 03:23:41.540482 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:41.540441    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffa8c11cfe276215bef10fec0e9aac1b909eeeb57cf3b1df7e750138be50ee1a"} err="failed to get container status \"ffa8c11cfe276215bef10fec0e9aac1b909eeeb57cf3b1df7e750138be50ee1a\": rpc error: code = NotFound desc = could not find container \"ffa8c11cfe276215bef10fec0e9aac1b909eeeb57cf3b1df7e750138be50ee1a\": container with ID starting with ffa8c11cfe276215bef10fec0e9aac1b909eeeb57cf3b1df7e750138be50ee1a not found: ID does not exist"
Sep 14 03:23:41.548541 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:41.548521    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"]
Sep 14 03:23:41.554711 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:41.554691    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3b049b-6f6df9f978-zgmkd"]
Sep 14 03:23:41.746146 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:41.746111    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7395698b-326f-4632-9eab-fde9abb0f8f8" path="/var/lib/kubelet/pods/7395698b-326f-4632-9eab-fde9abb0f8f8/volumes"
Sep 14 03:23:47.676580 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:47.676544    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"]
Sep 14 03:23:47.677154 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:47.676730    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt" podUID="c557a6a8-a252-47b5-bd0a-b37cf8ba1831" containerName="istio-proxy" containerID="cri-o://8e031b562f9ce6a6550dba3f35f3b553f6026197617cd03d1afa58d36df76fb8" gracePeriod=30
Sep 14 03:23:51.554031 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:51.553980    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw"]
Sep 14 03:23:51.554539 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:51.554473    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7395698b-326f-4632-9eab-fde9abb0f8f8" containerName="maas-api"
Sep 14 03:23:51.554539 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:51.554492    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7395698b-326f-4632-9eab-fde9abb0f8f8" containerName="maas-api"
Sep 14 03:23:51.554621 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:51.554599    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7395698b-326f-4632-9eab-fde9abb0f8f8" containerName="maas-api"
Sep 14 03:23:51.557617 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:51.557602    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw"
Sep 14 03:23:51.560472 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:51.560444    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:23:51.566154 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:51.566133    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw"]
Sep 14 03:23:51.681126 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:51.681086    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f316e6f5-62d9-4a8e-b460-cd3f790800d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw\" (UID: \"f316e6f5-62d9-4a8e-b460-cd3f790800d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw"
Sep 14 03:23:51.782158 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:51.782128    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f316e6f5-62d9-4a8e-b460-cd3f790800d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw\" (UID: \"f316e6f5-62d9-4a8e-b460-cd3f790800d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw"
Sep 14 03:23:51.782798 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:51.782774    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f316e6f5-62d9-4a8e-b460-cd3f790800d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw\" (UID: \"f316e6f5-62d9-4a8e-b460-cd3f790800d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw"
Sep 14 03:23:51.869224 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:51.869151    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw"
Sep 14 03:23:52.152670 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.152647    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw"]
Sep 14 03:23:52.154570 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:23:52.154545    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf316e6f5_62d9_4a8e_b460_cd3f790800d1.slice/crio-54ff38af977ed37810a090fddc0a077502fb8d86fc4b810cb42de8f744149c86 WatchSource:0}: Error finding container 54ff38af977ed37810a090fddc0a077502fb8d86fc4b810cb42de8f744149c86: Status 404 returned error can't find the container with id 54ff38af977ed37810a090fddc0a077502fb8d86fc4b810cb42de8f744149c86
Sep 14 03:23:52.586161 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.586126    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw" event={"ID":"f316e6f5-62d9-4a8e-b460-cd3f790800d1","Type":"ContainerStarted","Data":"6a75105428ac5a27523ca6c9209fd961b064ae7cb773e9af994ca7b09fca4f15"}
Sep 14 03:23:52.586563 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.586204    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw" event={"ID":"f316e6f5-62d9-4a8e-b460-cd3f790800d1","Type":"ContainerStarted","Data":"54ff38af977ed37810a090fddc0a077502fb8d86fc4b810cb42de8f744149c86"}
Sep 14 03:23:52.609897 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.609837    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw" podStartSLOduration=1.6098213179999998 podStartE2EDuration="1.609821318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:23:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:23:52.608752892 +0000 UTC m=+1895.394814805" watchObservedRunningTime="2026-09-14 03:23:52.609821318 +0000 UTC m=+1895.395883231"
Sep 14 03:23:52.925036 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.925014    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:23:52.993808 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.993783    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istiod-ca-cert\") pod \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") "
Sep 14 03:23:52.993968 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.993820    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-podinfo\") pod \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") "
Sep 14 03:23:52.993968 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.993845    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-token\") pod \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") "
Sep 14 03:23:52.993968 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.993873    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-workload-socket\") pod \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") "
Sep 14 03:23:52.994123 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.994035    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-data\") pod \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") "
Sep 14 03:23:52.994204 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.994125    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-workload-certs\") pod \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") "
Sep 14 03:23:52.994204 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.994125    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c557a6a8-a252-47b5-bd0a-b37cf8ba1831" (UID: "c557a6a8-a252-47b5-bd0a-b37cf8ba1831"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:23:52.994204 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.994164    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-envoy\") pod \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") "
Sep 14 03:23:52.994395 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.994218    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lv7c\" (UniqueName: \"kubernetes.io/projected/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-kube-api-access-5lv7c\") pod \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") "
Sep 14 03:23:52.994395 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.994282    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-credential-socket\") pod \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\" (UID: \"c557a6a8-a252-47b5-bd0a-b37cf8ba1831\") "
Sep 14 03:23:52.994395 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.994342    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-data" (OuterVolumeSpecName: "istio-data") pod "c557a6a8-a252-47b5-bd0a-b37cf8ba1831" (UID: "c557a6a8-a252-47b5-bd0a-b37cf8ba1831"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:23:52.994395 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.994365    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c557a6a8-a252-47b5-bd0a-b37cf8ba1831" (UID: "c557a6a8-a252-47b5-bd0a-b37cf8ba1831"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:23:52.994395 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.994368    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c557a6a8-a252-47b5-bd0a-b37cf8ba1831" (UID: "c557a6a8-a252-47b5-bd0a-b37cf8ba1831"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:23:52.994657 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.994579    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c557a6a8-a252-47b5-bd0a-b37cf8ba1831" (UID: "c557a6a8-a252-47b5-bd0a-b37cf8ba1831"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:23:52.994657 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.994633    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:23:52.994657 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.994654    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:23:52.994817 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.994669    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:23:52.994817 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.994682    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:23:52.996284 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.996260    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-token" (OuterVolumeSpecName: "istio-token") pod "c557a6a8-a252-47b5-bd0a-b37cf8ba1831" (UID: "c557a6a8-a252-47b5-bd0a-b37cf8ba1831"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:23:52.996478 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.996456    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-kube-api-access-5lv7c" (OuterVolumeSpecName: "kube-api-access-5lv7c") pod "c557a6a8-a252-47b5-bd0a-b37cf8ba1831" (UID: "c557a6a8-a252-47b5-bd0a-b37cf8ba1831"). InnerVolumeSpecName "kube-api-access-5lv7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:23:52.996912 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.996884    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c557a6a8-a252-47b5-bd0a-b37cf8ba1831" (UID: "c557a6a8-a252-47b5-bd0a-b37cf8ba1831"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:23:52.996912 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:52.996891    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c557a6a8-a252-47b5-bd0a-b37cf8ba1831" (UID: "c557a6a8-a252-47b5-bd0a-b37cf8ba1831"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:23:53.096103 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.096075    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:23:53.096103 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.096099    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lv7c\" (UniqueName: \"kubernetes.io/projected/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-kube-api-access-5lv7c\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:23:53.096323 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.096111    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:23:53.096323 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.096120    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:23:53.096323 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.096128    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c557a6a8-a252-47b5-bd0a-b37cf8ba1831-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:23:53.595215 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.595158    2579 generic.go:356] "Generic (PLEG): container finished" podID="c557a6a8-a252-47b5-bd0a-b37cf8ba1831" containerID="8e031b562f9ce6a6550dba3f35f3b553f6026197617cd03d1afa58d36df76fb8" exitCode=0
Sep 14 03:23:53.595675 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.595209    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt" event={"ID":"c557a6a8-a252-47b5-bd0a-b37cf8ba1831","Type":"ContainerDied","Data":"8e031b562f9ce6a6550dba3f35f3b553f6026197617cd03d1afa58d36df76fb8"}
Sep 14 03:23:53.595675 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.595237    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"
Sep 14 03:23:53.595675 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.595274    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt" event={"ID":"c557a6a8-a252-47b5-bd0a-b37cf8ba1831","Type":"ContainerDied","Data":"f2989a7f1a8e5abee37e2ce20e8b292c72ba6980ae752e415c3ed09073a927a5"}
Sep 14 03:23:53.595675 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.595305    2579 scope.go:117] "RemoveContainer" containerID="8e031b562f9ce6a6550dba3f35f3b553f6026197617cd03d1afa58d36df76fb8"
Sep 14 03:23:53.597157 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.597134    2579 generic.go:356] "Generic (PLEG): container finished" podID="f316e6f5-62d9-4a8e-b460-cd3f790800d1" containerID="6a75105428ac5a27523ca6c9209fd961b064ae7cb773e9af994ca7b09fca4f15" exitCode=0
Sep 14 03:23:53.597274 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.597206    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw" event={"ID":"f316e6f5-62d9-4a8e-b460-cd3f790800d1","Type":"ContainerDied","Data":"6a75105428ac5a27523ca6c9209fd961b064ae7cb773e9af994ca7b09fca4f15"}
Sep 14 03:23:53.604914 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.604900    2579 scope.go:117] "RemoveContainer" containerID="e7f4660ca9e852c5f502ace7ba7a3ab038d57e927458ac663b7108a560f0ff8c"
Sep 14 03:23:53.612943 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.612929    2579 scope.go:117] "RemoveContainer" containerID="8e031b562f9ce6a6550dba3f35f3b553f6026197617cd03d1afa58d36df76fb8"
Sep 14 03:23:53.613187 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:23:53.613154    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e031b562f9ce6a6550dba3f35f3b553f6026197617cd03d1afa58d36df76fb8\": container with ID starting with 8e031b562f9ce6a6550dba3f35f3b553f6026197617cd03d1afa58d36df76fb8 not found: ID does not exist" containerID="8e031b562f9ce6a6550dba3f35f3b553f6026197617cd03d1afa58d36df76fb8"
Sep 14 03:23:53.613254 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.613201    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e031b562f9ce6a6550dba3f35f3b553f6026197617cd03d1afa58d36df76fb8"} err="failed to get container status \"8e031b562f9ce6a6550dba3f35f3b553f6026197617cd03d1afa58d36df76fb8\": rpc error: code = NotFound desc = could not find container \"8e031b562f9ce6a6550dba3f35f3b553f6026197617cd03d1afa58d36df76fb8\": container with ID starting with 8e031b562f9ce6a6550dba3f35f3b553f6026197617cd03d1afa58d36df76fb8 not found: ID does not exist"
Sep 14 03:23:53.613254 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.613218    2579 scope.go:117] "RemoveContainer" containerID="e7f4660ca9e852c5f502ace7ba7a3ab038d57e927458ac663b7108a560f0ff8c"
Sep 14 03:23:53.613488 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:23:53.613471    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7f4660ca9e852c5f502ace7ba7a3ab038d57e927458ac663b7108a560f0ff8c\": container with ID starting with e7f4660ca9e852c5f502ace7ba7a3ab038d57e927458ac663b7108a560f0ff8c not found: ID does not exist" containerID="e7f4660ca9e852c5f502ace7ba7a3ab038d57e927458ac663b7108a560f0ff8c"
Sep 14 03:23:53.613539 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.613492    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7f4660ca9e852c5f502ace7ba7a3ab038d57e927458ac663b7108a560f0ff8c"} err="failed to get container status \"e7f4660ca9e852c5f502ace7ba7a3ab038d57e927458ac663b7108a560f0ff8c\": rpc error: code = NotFound desc = could not find container \"e7f4660ca9e852c5f502ace7ba7a3ab038d57e927458ac663b7108a560f0ff8c\": container with ID starting with e7f4660ca9e852c5f502ace7ba7a3ab038d57e927458ac663b7108a560f0ff8c not found: ID does not exist"
Sep 14 03:23:53.632971 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.632946    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"]
Sep 14 03:23:53.643608 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.643585    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-3b049b-openshift-default-d97f57556-cgdgt"]
Sep 14 03:23:53.743866 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:53.743834    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c557a6a8-a252-47b5-bd0a-b37cf8ba1831" path="/var/lib/kubelet/pods/c557a6a8-a252-47b5-bd0a-b37cf8ba1831/volumes"
Sep 14 03:23:54.741009 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:54.740987    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw"
Sep 14 03:23:54.811188 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:54.811157    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f316e6f5-62d9-4a8e-b460-cd3f790800d1-maas-api-service-ca\") pod \"f316e6f5-62d9-4a8e-b460-cd3f790800d1\" (UID: \"f316e6f5-62d9-4a8e-b460-cd3f790800d1\") "
Sep 14 03:23:54.811519 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:54.811497    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f316e6f5-62d9-4a8e-b460-cd3f790800d1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f316e6f5-62d9-4a8e-b460-cd3f790800d1" (UID: "f316e6f5-62d9-4a8e-b460-cd3f790800d1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:23:54.912108 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:54.912040    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f316e6f5-62d9-4a8e-b460-cd3f790800d1-maas-api-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:23:55.609804 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:55.609768    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw" event={"ID":"f316e6f5-62d9-4a8e-b460-cd3f790800d1","Type":"ContainerDied","Data":"54ff38af977ed37810a090fddc0a077502fb8d86fc4b810cb42de8f744149c86"}
Sep 14 03:23:55.610016 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:55.609821    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54ff38af977ed37810a090fddc0a077502fb8d86fc4b810cb42de8f744149c86"
Sep 14 03:23:55.610016 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:23:55.609782    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw"
Sep 14 03:24:07.154637 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.154605    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"]
Sep 14 03:24:07.155095 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.154824    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c" podUID="9f17d7d4-7aba-47fd-80f8-5513f4645b98" containerName="payload-pre-processing" containerID="cri-o://d8c9bf990bbcc2eb90d1d12b01be2df30f060bde86c49534cf135fc8464cafe9" gracePeriod=30
Sep 14 03:24:07.422032 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.422008    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"
Sep 14 03:24:07.494942 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.494893    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw"]
Sep 14 03:24:07.497697 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.497673    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d081a3-htksw"]
Sep 14 03:24:07.530783 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.530755    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f17d7d4-7aba-47fd-80f8-5513f4645b98-plugins-config-volume\") pod \"9f17d7d4-7aba-47fd-80f8-5513f4645b98\" (UID: \"9f17d7d4-7aba-47fd-80f8-5513f4645b98\") "
Sep 14 03:24:07.530924 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.530800    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvxpm\" (UniqueName: \"kubernetes.io/projected/9f17d7d4-7aba-47fd-80f8-5513f4645b98-kube-api-access-bvxpm\") pod \"9f17d7d4-7aba-47fd-80f8-5513f4645b98\" (UID: \"9f17d7d4-7aba-47fd-80f8-5513f4645b98\") "
Sep 14 03:24:07.531163 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.531138    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f17d7d4-7aba-47fd-80f8-5513f4645b98-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9f17d7d4-7aba-47fd-80f8-5513f4645b98" (UID: "9f17d7d4-7aba-47fd-80f8-5513f4645b98"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:24:07.532997 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.532978    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f17d7d4-7aba-47fd-80f8-5513f4645b98-kube-api-access-bvxpm" (OuterVolumeSpecName: "kube-api-access-bvxpm") pod "9f17d7d4-7aba-47fd-80f8-5513f4645b98" (UID: "9f17d7d4-7aba-47fd-80f8-5513f4645b98"). InnerVolumeSpecName "kube-api-access-bvxpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:24:07.632196 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.632142    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvxpm\" (UniqueName: \"kubernetes.io/projected/9f17d7d4-7aba-47fd-80f8-5513f4645b98-kube-api-access-bvxpm\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:24:07.632196 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.632168    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f17d7d4-7aba-47fd-80f8-5513f4645b98-plugins-config-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:24:07.673321 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.673241    2579 generic.go:356] "Generic (PLEG): container finished" podID="9f17d7d4-7aba-47fd-80f8-5513f4645b98" containerID="d8c9bf990bbcc2eb90d1d12b01be2df30f060bde86c49534cf135fc8464cafe9" exitCode=0
Sep 14 03:24:07.673321 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.673283    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"
Sep 14 03:24:07.673523 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.673283    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c" event={"ID":"9f17d7d4-7aba-47fd-80f8-5513f4645b98","Type":"ContainerDied","Data":"d8c9bf990bbcc2eb90d1d12b01be2df30f060bde86c49534cf135fc8464cafe9"}
Sep 14 03:24:07.673523 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.673408    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c" event={"ID":"9f17d7d4-7aba-47fd-80f8-5513f4645b98","Type":"ContainerDied","Data":"990d65ad57204427ce51198fae247851a9c1c5c41650efc97071217d7a6fce5b"}
Sep 14 03:24:07.673523 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.673435    2579 scope.go:117] "RemoveContainer" containerID="d8c9bf990bbcc2eb90d1d12b01be2df30f060bde86c49534cf135fc8464cafe9"
Sep 14 03:24:07.682264 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.682248    2579 scope.go:117] "RemoveContainer" containerID="d8c9bf990bbcc2eb90d1d12b01be2df30f060bde86c49534cf135fc8464cafe9"
Sep 14 03:24:07.682552 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:24:07.682535    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8c9bf990bbcc2eb90d1d12b01be2df30f060bde86c49534cf135fc8464cafe9\": container with ID starting with d8c9bf990bbcc2eb90d1d12b01be2df30f060bde86c49534cf135fc8464cafe9 not found: ID does not exist" containerID="d8c9bf990bbcc2eb90d1d12b01be2df30f060bde86c49534cf135fc8464cafe9"
Sep 14 03:24:07.682624 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.682564    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8c9bf990bbcc2eb90d1d12b01be2df30f060bde86c49534cf135fc8464cafe9"} err="failed to get container status \"d8c9bf990bbcc2eb90d1d12b01be2df30f060bde86c49534cf135fc8464cafe9\": rpc error: code = NotFound desc = could not find container \"d8c9bf990bbcc2eb90d1d12b01be2df30f060bde86c49534cf135fc8464cafe9\": container with ID starting with d8c9bf990bbcc2eb90d1d12b01be2df30f060bde86c49534cf135fc8464cafe9 not found: ID does not exist"
Sep 14 03:24:07.698018 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.697993    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"]
Sep 14 03:24:07.701696 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.701672    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d081a3-57475879d6-j954c"]
Sep 14 03:24:07.743654 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.743625    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f17d7d4-7aba-47fd-80f8-5513f4645b98" path="/var/lib/kubelet/pods/9f17d7d4-7aba-47fd-80f8-5513f4645b98/volumes"
Sep 14 03:24:07.743958 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:07.743945    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f316e6f5-62d9-4a8e-b460-cd3f790800d1" path="/var/lib/kubelet/pods/f316e6f5-62d9-4a8e-b460-cd3f790800d1/volumes"
Sep 14 03:24:18.135982 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:18.135950    2579 scope.go:117] "RemoveContainer" containerID="f91532f21a5cb546279c9503b766e2692d16c9bcb0b32b8ebfc4165fcbf6e1ac"
Sep 14 03:24:18.144115 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:18.144091    2579 scope.go:117] "RemoveContainer" containerID="d0c15079c41f9a0581b845c5559bd3b869b27d3704687b76777be29ff04550ae"
Sep 14 03:24:18.151851 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:18.151834    2579 scope.go:117] "RemoveContainer" containerID="4c1f158de80b497fdf52f93fe02549eed9706debeded0c1a62fbfe7e5c413ecf"
Sep 14 03:24:56.651716 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.651676    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb"]
Sep 14 03:24:56.652284 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.652214    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c557a6a8-a252-47b5-bd0a-b37cf8ba1831" containerName="istio-proxy"
Sep 14 03:24:56.652284 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.652234    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c557a6a8-a252-47b5-bd0a-b37cf8ba1831" containerName="istio-proxy"
Sep 14 03:24:56.652284 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.652246    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f316e6f5-62d9-4a8e-b460-cd3f790800d1" containerName="revoke-keys"
Sep 14 03:24:56.652284 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.652255    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f316e6f5-62d9-4a8e-b460-cd3f790800d1" containerName="revoke-keys"
Sep 14 03:24:56.652514 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.652289    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f17d7d4-7aba-47fd-80f8-5513f4645b98" containerName="payload-pre-processing"
Sep 14 03:24:56.652514 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.652298    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f17d7d4-7aba-47fd-80f8-5513f4645b98" containerName="payload-pre-processing"
Sep 14 03:24:56.652514 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.652417    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c557a6a8-a252-47b5-bd0a-b37cf8ba1831" containerName="istio-proxy"
Sep 14 03:24:56.652514 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.652430    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c557a6a8-a252-47b5-bd0a-b37cf8ba1831" containerName="istio-proxy"
Sep 14 03:24:56.652514 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.652441    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f316e6f5-62d9-4a8e-b460-cd3f790800d1" containerName="revoke-keys"
Sep 14 03:24:56.652514 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.652452    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f17d7d4-7aba-47fd-80f8-5513f4645b98" containerName="payload-pre-processing"
Sep 14 03:24:56.655608 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.655587    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb"
Sep 14 03:24:56.658505 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.658488    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:24:56.667288 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.667268    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb"]
Sep 14 03:24:56.729692 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.729665    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb\" (UID: \"d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb"
Sep 14 03:24:56.830979 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.830943    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb\" (UID: \"d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb"
Sep 14 03:24:56.831620 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.831585    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb\" (UID: \"d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb"
Sep 14 03:24:56.966300 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:56.966226    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb"
Sep 14 03:24:57.242138 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:57.242112    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb"]
Sep 14 03:24:57.244475 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:24:57.244448    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4aafe5e_4436_4d1e_bd7c_54ea1a9bd33b.slice/crio-ae4e6d374cf6570aafd02cf6f6f428b6b08728b54bc9f50ab35359d7c62a8dac WatchSource:0}: Error finding container ae4e6d374cf6570aafd02cf6f6f428b6b08728b54bc9f50ab35359d7c62a8dac: Status 404 returned error can't find the container with id ae4e6d374cf6570aafd02cf6f6f428b6b08728b54bc9f50ab35359d7c62a8dac
Sep 14 03:24:57.919214 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:57.919103    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb" event={"ID":"d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b","Type":"ContainerStarted","Data":"9d2a2d7b830a3f607a81dee5f1b0e47cf4287d09fd4d2091705a61762876950a"}
Sep 14 03:24:57.919214 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:57.919169    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb" event={"ID":"d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b","Type":"ContainerStarted","Data":"ae4e6d374cf6570aafd02cf6f6f428b6b08728b54bc9f50ab35359d7c62a8dac"}
Sep 14 03:24:57.936662 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:57.936614    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb" podStartSLOduration=1.936599993 podStartE2EDuration="1.936599993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:24:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:24:57.934626215 +0000 UTC m=+1960.720688131" watchObservedRunningTime="2026-09-14 03:24:57.936599993 +0000 UTC m=+1960.722661906"
Sep 14 03:24:58.925876 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:58.925835    2579 generic.go:356] "Generic (PLEG): container finished" podID="d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b" containerID="9d2a2d7b830a3f607a81dee5f1b0e47cf4287d09fd4d2091705a61762876950a" exitCode=0
Sep 14 03:24:58.926265 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:24:58.925923    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb" event={"ID":"d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b","Type":"ContainerDied","Data":"9d2a2d7b830a3f607a81dee5f1b0e47cf4287d09fd4d2091705a61762876950a"}
Sep 14 03:25:00.061874 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:00.061846    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb"
Sep 14 03:25:00.163342 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:00.163315    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b-maas-api-service-ca\") pod \"d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b\" (UID: \"d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b\") "
Sep 14 03:25:00.163626 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:00.163606    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b" (UID: "d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:25:00.264693 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:00.264662    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b-maas-api-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:00.936667 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:00.936637    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb"
Sep 14 03:25:00.936667 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:00.936641    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb" event={"ID":"d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b","Type":"ContainerDied","Data":"ae4e6d374cf6570aafd02cf6f6f428b6b08728b54bc9f50ab35359d7c62a8dac"}
Sep 14 03:25:00.936941 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:00.936684    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae4e6d374cf6570aafd02cf6f6f428b6b08728b54bc9f50ab35359d7c62a8dac"
Sep 14 03:25:07.374445 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:07.374401    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb"]
Sep 14 03:25:07.376476 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:07.376453    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8d29d7-96hlb"]
Sep 14 03:25:07.743652 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:07.743626    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b" path="/var/lib/kubelet/pods/d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b/volumes"
Sep 14 03:25:24.960503 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:24.960415    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"]
Sep 14 03:25:24.960926 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:24.960669    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s" podUID="cc167e10-f611-4ff9-a565-9359ee291d76" containerName="maas-api" containerID="cri-o://30a4e3146357e468a9fe5000a65ab53e855a2a2b82bf8ecb14fdc074d9af1e4f" gracePeriod=30
Sep 14 03:25:25.056971 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.056936    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"]
Sep 14 03:25:25.057275 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.057245    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl" podUID="ee8688c3-0edf-42bb-8157-c8100d5c53c9" containerName="payload-processing" containerID="cri-o://7fccde9b8aff9c5d6a1906b0f4368ae0806fd1376f6e45c92cf70f98c5ee81e5" gracePeriod=30
Sep 14 03:25:25.226774 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.226748    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:25:25.276491 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.276463    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc167e10-f611-4ff9-a565-9359ee291d76-metrics-certs\") pod \"cc167e10-f611-4ff9-a565-9359ee291d76\" (UID: \"cc167e10-f611-4ff9-a565-9359ee291d76\") "
Sep 14 03:25:25.276696 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.276533    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc167e10-f611-4ff9-a565-9359ee291d76-openshift-service-ca\") pod \"cc167e10-f611-4ff9-a565-9359ee291d76\" (UID: \"cc167e10-f611-4ff9-a565-9359ee291d76\") "
Sep 14 03:25:25.276696 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.276602    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4bkp\" (UniqueName: \"kubernetes.io/projected/cc167e10-f611-4ff9-a565-9359ee291d76-kube-api-access-l4bkp\") pod \"cc167e10-f611-4ff9-a565-9359ee291d76\" (UID: \"cc167e10-f611-4ff9-a565-9359ee291d76\") "
Sep 14 03:25:25.276696 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.276693    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc167e10-f611-4ff9-a565-9359ee291d76-maas-api-tls\") pod \"cc167e10-f611-4ff9-a565-9359ee291d76\" (UID: \"cc167e10-f611-4ff9-a565-9359ee291d76\") "
Sep 14 03:25:25.276933 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.276907    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc167e10-f611-4ff9-a565-9359ee291d76-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cc167e10-f611-4ff9-a565-9359ee291d76" (UID: "cc167e10-f611-4ff9-a565-9359ee291d76"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:25:25.279977 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.279934    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc167e10-f611-4ff9-a565-9359ee291d76-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cc167e10-f611-4ff9-a565-9359ee291d76" (UID: "cc167e10-f611-4ff9-a565-9359ee291d76"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:25:25.280098 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.280027    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc167e10-f611-4ff9-a565-9359ee291d76-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cc167e10-f611-4ff9-a565-9359ee291d76" (UID: "cc167e10-f611-4ff9-a565-9359ee291d76"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:25:25.280098 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.280030    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc167e10-f611-4ff9-a565-9359ee291d76-kube-api-access-l4bkp" (OuterVolumeSpecName: "kube-api-access-l4bkp") pod "cc167e10-f611-4ff9-a565-9359ee291d76" (UID: "cc167e10-f611-4ff9-a565-9359ee291d76"). InnerVolumeSpecName "kube-api-access-l4bkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:25:25.305754 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.305733    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"
Sep 14 03:25:25.377679 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.377650    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqv8l\" (UniqueName: \"kubernetes.io/projected/ee8688c3-0edf-42bb-8157-c8100d5c53c9-kube-api-access-hqv8l\") pod \"ee8688c3-0edf-42bb-8157-c8100d5c53c9\" (UID: \"ee8688c3-0edf-42bb-8157-c8100d5c53c9\") "
Sep 14 03:25:25.377879 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.377741    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee8688c3-0edf-42bb-8157-c8100d5c53c9-plugins-config-volume\") pod \"ee8688c3-0edf-42bb-8157-c8100d5c53c9\" (UID: \"ee8688c3-0edf-42bb-8157-c8100d5c53c9\") "
Sep 14 03:25:25.377954 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.377915    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc167e10-f611-4ff9-a565-9359ee291d76-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:25.377954 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.377926    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc167e10-f611-4ff9-a565-9359ee291d76-metrics-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:25.377954 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.377936    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc167e10-f611-4ff9-a565-9359ee291d76-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:25.377954 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.377944    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4bkp\" (UniqueName: \"kubernetes.io/projected/cc167e10-f611-4ff9-a565-9359ee291d76-kube-api-access-l4bkp\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:25.378213 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.378165    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee8688c3-0edf-42bb-8157-c8100d5c53c9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ee8688c3-0edf-42bb-8157-c8100d5c53c9" (UID: "ee8688c3-0edf-42bb-8157-c8100d5c53c9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:25:25.381631 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.381605    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee8688c3-0edf-42bb-8157-c8100d5c53c9-kube-api-access-hqv8l" (OuterVolumeSpecName: "kube-api-access-hqv8l") pod "ee8688c3-0edf-42bb-8157-c8100d5c53c9" (UID: "ee8688c3-0edf-42bb-8157-c8100d5c53c9"). InnerVolumeSpecName "kube-api-access-hqv8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:25:25.479316 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.479209    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqv8l\" (UniqueName: \"kubernetes.io/projected/ee8688c3-0edf-42bb-8157-c8100d5c53c9-kube-api-access-hqv8l\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:25.479316 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:25.479255    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee8688c3-0edf-42bb-8157-c8100d5c53c9-plugins-config-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:26.061055 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.060979    2579 generic.go:356] "Generic (PLEG): container finished" podID="ee8688c3-0edf-42bb-8157-c8100d5c53c9" containerID="7fccde9b8aff9c5d6a1906b0f4368ae0806fd1376f6e45c92cf70f98c5ee81e5" exitCode=0
Sep 14 03:25:26.061055 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.061025    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"
Sep 14 03:25:26.061550 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.061068    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl" event={"ID":"ee8688c3-0edf-42bb-8157-c8100d5c53c9","Type":"ContainerDied","Data":"7fccde9b8aff9c5d6a1906b0f4368ae0806fd1376f6e45c92cf70f98c5ee81e5"}
Sep 14 03:25:26.061550 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.061123    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl" event={"ID":"ee8688c3-0edf-42bb-8157-c8100d5c53c9","Type":"ContainerDied","Data":"66aa8c33064bf25f1c7882ac745a1934c95e79e37da3b65bc7fa2222143a4166"}
Sep 14 03:25:26.061550 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.061154    2579 scope.go:117] "RemoveContainer" containerID="7fccde9b8aff9c5d6a1906b0f4368ae0806fd1376f6e45c92cf70f98c5ee81e5"
Sep 14 03:25:26.062602 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.062582    2579 generic.go:356] "Generic (PLEG): container finished" podID="cc167e10-f611-4ff9-a565-9359ee291d76" containerID="30a4e3146357e468a9fe5000a65ab53e855a2a2b82bf8ecb14fdc074d9af1e4f" exitCode=0
Sep 14 03:25:26.062691 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.062623    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s" event={"ID":"cc167e10-f611-4ff9-a565-9359ee291d76","Type":"ContainerDied","Data":"30a4e3146357e468a9fe5000a65ab53e855a2a2b82bf8ecb14fdc074d9af1e4f"}
Sep 14 03:25:26.062691 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.062641    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"
Sep 14 03:25:26.062691 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.062662    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s" event={"ID":"cc167e10-f611-4ff9-a565-9359ee291d76","Type":"ContainerDied","Data":"a8aa6a19fe9df49ad9d30dadd12fd7a5a5b1dc20928c03168b536a07fb22c8bb"}
Sep 14 03:25:26.069807 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.069788    2579 scope.go:117] "RemoveContainer" containerID="7fccde9b8aff9c5d6a1906b0f4368ae0806fd1376f6e45c92cf70f98c5ee81e5"
Sep 14 03:25:26.070092 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:25:26.070072    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fccde9b8aff9c5d6a1906b0f4368ae0806fd1376f6e45c92cf70f98c5ee81e5\": container with ID starting with 7fccde9b8aff9c5d6a1906b0f4368ae0806fd1376f6e45c92cf70f98c5ee81e5 not found: ID does not exist" containerID="7fccde9b8aff9c5d6a1906b0f4368ae0806fd1376f6e45c92cf70f98c5ee81e5"
Sep 14 03:25:26.070196 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.070101    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fccde9b8aff9c5d6a1906b0f4368ae0806fd1376f6e45c92cf70f98c5ee81e5"} err="failed to get container status \"7fccde9b8aff9c5d6a1906b0f4368ae0806fd1376f6e45c92cf70f98c5ee81e5\": rpc error: code = NotFound desc = could not find container \"7fccde9b8aff9c5d6a1906b0f4368ae0806fd1376f6e45c92cf70f98c5ee81e5\": container with ID starting with 7fccde9b8aff9c5d6a1906b0f4368ae0806fd1376f6e45c92cf70f98c5ee81e5 not found: ID does not exist"
Sep 14 03:25:26.070196 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.070121    2579 scope.go:117] "RemoveContainer" containerID="30a4e3146357e468a9fe5000a65ab53e855a2a2b82bf8ecb14fdc074d9af1e4f"
Sep 14 03:25:26.077936 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.077919    2579 scope.go:117] "RemoveContainer" containerID="30a4e3146357e468a9fe5000a65ab53e855a2a2b82bf8ecb14fdc074d9af1e4f"
Sep 14 03:25:26.078406 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:25:26.078380    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30a4e3146357e468a9fe5000a65ab53e855a2a2b82bf8ecb14fdc074d9af1e4f\": container with ID starting with 30a4e3146357e468a9fe5000a65ab53e855a2a2b82bf8ecb14fdc074d9af1e4f not found: ID does not exist" containerID="30a4e3146357e468a9fe5000a65ab53e855a2a2b82bf8ecb14fdc074d9af1e4f"
Sep 14 03:25:26.078486 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.078427    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30a4e3146357e468a9fe5000a65ab53e855a2a2b82bf8ecb14fdc074d9af1e4f"} err="failed to get container status \"30a4e3146357e468a9fe5000a65ab53e855a2a2b82bf8ecb14fdc074d9af1e4f\": rpc error: code = NotFound desc = could not find container \"30a4e3146357e468a9fe5000a65ab53e855a2a2b82bf8ecb14fdc074d9af1e4f\": container with ID starting with 30a4e3146357e468a9fe5000a65ab53e855a2a2b82bf8ecb14fdc074d9af1e4f not found: ID does not exist"
Sep 14 03:25:26.080101 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.080083    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"]
Sep 14 03:25:26.086830 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.086811    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2b32aa-5959d5bf89-hbzbl"]
Sep 14 03:25:26.097103 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.097083    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"]
Sep 14 03:25:26.102768 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:26.102750    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2b32aa-685c75bc47-prr9s"]
Sep 14 03:25:27.743875 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:27.743840    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc167e10-f611-4ff9-a565-9359ee291d76" path="/var/lib/kubelet/pods/cc167e10-f611-4ff9-a565-9359ee291d76/volumes"
Sep 14 03:25:27.744352 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:27.744337    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee8688c3-0edf-42bb-8157-c8100d5c53c9" path="/var/lib/kubelet/pods/ee8688c3-0edf-42bb-8157-c8100d5c53c9/volumes"
Sep 14 03:25:32.930329 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:32.930297    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"]
Sep 14 03:25:32.930742 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:32.930492    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm" podUID="5da2703a-da04-4553-ad09-b76929116540" containerName="istio-proxy" containerID="cri-o://d5b979eb1fa3270356c5e2f4d820134a8c43ccf8d62f30728177d0510f5b734c" gracePeriod=30
Sep 14 03:25:33.339759 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.339730    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8"]
Sep 14 03:25:33.340046 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.340035    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc167e10-f611-4ff9-a565-9359ee291d76" containerName="maas-api"
Sep 14 03:25:33.340092 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.340061    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc167e10-f611-4ff9-a565-9359ee291d76" containerName="maas-api"
Sep 14 03:25:33.340092 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.340078    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c557a6a8-a252-47b5-bd0a-b37cf8ba1831" containerName="istio-proxy"
Sep 14 03:25:33.340092 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.340083    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c557a6a8-a252-47b5-bd0a-b37cf8ba1831" containerName="istio-proxy"
Sep 14 03:25:33.340207 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.340093    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee8688c3-0edf-42bb-8157-c8100d5c53c9" containerName="payload-processing"
Sep 14 03:25:33.340207 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.340098    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee8688c3-0edf-42bb-8157-c8100d5c53c9" containerName="payload-processing"
Sep 14 03:25:33.340207 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.340104    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b" containerName="revoke-keys"
Sep 14 03:25:33.340207 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.340110    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b" containerName="revoke-keys"
Sep 14 03:25:33.340337 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.340211    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc167e10-f611-4ff9-a565-9359ee291d76" containerName="maas-api"
Sep 14 03:25:33.340337 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.340220    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee8688c3-0edf-42bb-8157-c8100d5c53c9" containerName="payload-processing"
Sep 14 03:25:33.340337 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.340227    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4aafe5e-4436-4d1e-bd7c-54ea1a9bd33b" containerName="revoke-keys"
Sep 14 03:25:33.344762 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.344742    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8"
Sep 14 03:25:33.347929 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.347910    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:25:33.350447 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.350428    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:25:33.379147 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.379124    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8"]
Sep 14 03:25:33.444346 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.444317    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575c0e79-4bd5-4f3b-af3a-f4f62212745f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8\" (UID: \"575c0e79-4bd5-4f3b-af3a-f4f62212745f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8"
Sep 14 03:25:33.544858 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.544833    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575c0e79-4bd5-4f3b-af3a-f4f62212745f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8\" (UID: \"575c0e79-4bd5-4f3b-af3a-f4f62212745f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8"
Sep 14 03:25:33.545399 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.545383    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575c0e79-4bd5-4f3b-af3a-f4f62212745f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8\" (UID: \"575c0e79-4bd5-4f3b-af3a-f4f62212745f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8"
Sep 14 03:25:33.655084 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.655028    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8"
Sep 14 03:25:33.940283 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:33.940253    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8"]
Sep 14 03:25:33.942873 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:25:33.942847    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod575c0e79_4bd5_4f3b_af3a_f4f62212745f.slice/crio-91cf2d4916d4a9bc6d4b5e9a1008f22fbde0e2060463ab5c09fc1927eb41e713 WatchSource:0}: Error finding container 91cf2d4916d4a9bc6d4b5e9a1008f22fbde0e2060463ab5c09fc1927eb41e713: Status 404 returned error can't find the container with id 91cf2d4916d4a9bc6d4b5e9a1008f22fbde0e2060463ab5c09fc1927eb41e713
Sep 14 03:25:34.108220 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:34.108164    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8" event={"ID":"575c0e79-4bd5-4f3b-af3a-f4f62212745f","Type":"ContainerStarted","Data":"91cf2d4916d4a9bc6d4b5e9a1008f22fbde0e2060463ab5c09fc1927eb41e713"}
Sep 14 03:25:35.117411 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:35.117381    2579 generic.go:356] "Generic (PLEG): container finished" podID="575c0e79-4bd5-4f3b-af3a-f4f62212745f" containerID="20707cedfd7c55a94997901b190f33dcff728abe69ba7a543d349954736dfacf" exitCode=0
Sep 14 03:25:35.117814 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:35.117475    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8" event={"ID":"575c0e79-4bd5-4f3b-af3a-f4f62212745f","Type":"ContainerDied","Data":"20707cedfd7c55a94997901b190f33dcff728abe69ba7a543d349954736dfacf"}
Sep 14 03:25:36.250593 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:36.250569    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8"
Sep 14 03:25:36.367919 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:36.367891    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575c0e79-4bd5-4f3b-af3a-f4f62212745f-maas-api-service-ca\") pod \"575c0e79-4bd5-4f3b-af3a-f4f62212745f\" (UID: \"575c0e79-4bd5-4f3b-af3a-f4f62212745f\") "
Sep 14 03:25:36.368291 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:36.368270    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/575c0e79-4bd5-4f3b-af3a-f4f62212745f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "575c0e79-4bd5-4f3b-af3a-f4f62212745f" (UID: "575c0e79-4bd5-4f3b-af3a-f4f62212745f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:25:36.469412 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:36.469338    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575c0e79-4bd5-4f3b-af3a-f4f62212745f-maas-api-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:37.128295 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:37.128269    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8"
Sep 14 03:25:37.128473 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:37.128268    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8" event={"ID":"575c0e79-4bd5-4f3b-af3a-f4f62212745f","Type":"ContainerDied","Data":"91cf2d4916d4a9bc6d4b5e9a1008f22fbde0e2060463ab5c09fc1927eb41e713"}
Sep 14 03:25:37.128473 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:37.128395    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91cf2d4916d4a9bc6d4b5e9a1008f22fbde0e2060463ab5c09fc1927eb41e713"
Sep 14 03:25:37.276057 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:37.276027    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 03:25:37.276417 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:37.276087    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm" podUID="5da2703a-da04-4553-ad09-b76929116540" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 03:25:38.134966 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.134937    2579 generic.go:356] "Generic (PLEG): container finished" podID="5da2703a-da04-4553-ad09-b76929116540" containerID="d5b979eb1fa3270356c5e2f4d820134a8c43ccf8d62f30728177d0510f5b734c" exitCode=0
Sep 14 03:25:38.135094 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.134976    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm" event={"ID":"5da2703a-da04-4553-ad09-b76929116540","Type":"ContainerDied","Data":"d5b979eb1fa3270356c5e2f4d820134a8c43ccf8d62f30728177d0510f5b734c"}
Sep 14 03:25:38.135094 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.135029    2579 scope.go:117] "RemoveContainer" containerID="d407408ab6137c8fc884808010aafab1cf97483928af25a3175a066b7587dcd5"
Sep 14 03:25:38.189908 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.189883    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:25:38.283431 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.283404    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-workload-certs\") pod \"5da2703a-da04-4553-ad09-b76929116540\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") "
Sep 14 03:25:38.283818 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.283440    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5da2703a-da04-4553-ad09-b76929116540-istio-podinfo\") pod \"5da2703a-da04-4553-ad09-b76929116540\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") "
Sep 14 03:25:38.283818 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.283498    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-workload-socket\") pod \"5da2703a-da04-4553-ad09-b76929116540\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") "
Sep 14 03:25:38.283818 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.283556    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dmcs\" (UniqueName: \"kubernetes.io/projected/5da2703a-da04-4553-ad09-b76929116540-kube-api-access-8dmcs\") pod \"5da2703a-da04-4553-ad09-b76929116540\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") "
Sep 14 03:25:38.283818 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.283589    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-istio-envoy\") pod \"5da2703a-da04-4553-ad09-b76929116540\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") "
Sep 14 03:25:38.283818 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.283619    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-istio-data\") pod \"5da2703a-da04-4553-ad09-b76929116540\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") "
Sep 14 03:25:38.283818 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.283635    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5da2703a-da04-4553-ad09-b76929116540" (UID: "5da2703a-da04-4553-ad09-b76929116540"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:25:38.283818 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.283653    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5da2703a-da04-4553-ad09-b76929116540-istiod-ca-cert\") pod \"5da2703a-da04-4553-ad09-b76929116540\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") "
Sep 14 03:25:38.283818 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.283678    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-credential-socket\") pod \"5da2703a-da04-4553-ad09-b76929116540\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") "
Sep 14 03:25:38.283818 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.283745    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5da2703a-da04-4553-ad09-b76929116540-istio-token\") pod \"5da2703a-da04-4553-ad09-b76929116540\" (UID: \"5da2703a-da04-4553-ad09-b76929116540\") "
Sep 14 03:25:38.284304 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.283809    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5da2703a-da04-4553-ad09-b76929116540" (UID: "5da2703a-da04-4553-ad09-b76929116540"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:25:38.284304 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.283878    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-istio-data" (OuterVolumeSpecName: "istio-data") pod "5da2703a-da04-4553-ad09-b76929116540" (UID: "5da2703a-da04-4553-ad09-b76929116540"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:25:38.284304 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.283995    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5da2703a-da04-4553-ad09-b76929116540" (UID: "5da2703a-da04-4553-ad09-b76929116540"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:25:38.284304 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.284070    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5da2703a-da04-4553-ad09-b76929116540-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5da2703a-da04-4553-ad09-b76929116540" (UID: "5da2703a-da04-4553-ad09-b76929116540"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:25:38.284304 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.284075    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:38.284304 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.284136    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:38.284304 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.284147    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:38.284304 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.284163    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:38.285867 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.285838    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5da2703a-da04-4553-ad09-b76929116540" (UID: "5da2703a-da04-4553-ad09-b76929116540"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:25:38.286002 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.285983    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5da2703a-da04-4553-ad09-b76929116540-istio-token" (OuterVolumeSpecName: "istio-token") pod "5da2703a-da04-4553-ad09-b76929116540" (UID: "5da2703a-da04-4553-ad09-b76929116540"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:25:38.286309 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.286279    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5da2703a-da04-4553-ad09-b76929116540-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5da2703a-da04-4553-ad09-b76929116540" (UID: "5da2703a-da04-4553-ad09-b76929116540"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:25:38.286309 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.286288    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5da2703a-da04-4553-ad09-b76929116540-kube-api-access-8dmcs" (OuterVolumeSpecName: "kube-api-access-8dmcs") pod "5da2703a-da04-4553-ad09-b76929116540" (UID: "5da2703a-da04-4553-ad09-b76929116540"). InnerVolumeSpecName "kube-api-access-8dmcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:25:38.384839 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.384813    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dmcs\" (UniqueName: \"kubernetes.io/projected/5da2703a-da04-4553-ad09-b76929116540-kube-api-access-8dmcs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:38.385013 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.384840    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5da2703a-da04-4553-ad09-b76929116540-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:38.385013 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.384853    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5da2703a-da04-4553-ad09-b76929116540-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:38.385013 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.384864    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5da2703a-da04-4553-ad09-b76929116540-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:38.385013 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:38.384876    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5da2703a-da04-4553-ad09-b76929116540-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:39.141700 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:39.141671    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"
Sep 14 03:25:39.141888 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:39.141673    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm" event={"ID":"5da2703a-da04-4553-ad09-b76929116540","Type":"ContainerDied","Data":"e4b6a7718340008b118111121ad0c557df2cd8eadecf38687faeb1bc559de8c8"}
Sep 14 03:25:39.141888 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:39.141808    2579 scope.go:117] "RemoveContainer" containerID="d5b979eb1fa3270356c5e2f4d820134a8c43ccf8d62f30728177d0510f5b734c"
Sep 14 03:25:39.168577 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:39.168554    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"]
Sep 14 03:25:39.172434 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:39.172415    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2b32aa-openshift-default-7bd57545c4-l5dfm"]
Sep 14 03:25:39.744721 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:39.744685    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5da2703a-da04-4553-ad09-b76929116540" path="/var/lib/kubelet/pods/5da2703a-da04-4553-ad09-b76929116540/volumes"
Sep 14 03:25:43.957457 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:43.957419    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-177593-29822595-nt4d8"]
Sep 14 03:25:43.960654 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:43.960629    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-177593-29822595-nt4d8"]
Sep 14 03:25:44.299062 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:44.299028    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8"]
Sep 14 03:25:44.303617 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:44.303592    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-177593-cvbv8"]
Sep 14 03:25:45.743020 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:45.742984    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="065da8a6-6109-4130-9955-0334ca40c0a6" path="/var/lib/kubelet/pods/065da8a6-6109-4130-9955-0334ca40c0a6/volumes"
Sep 14 03:25:45.743403 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:45.743322    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="575c0e79-4bd5-4f3b-af3a-f4f62212745f" path="/var/lib/kubelet/pods/575c0e79-4bd5-4f3b-af3a-f4f62212745f/volumes"
Sep 14 03:25:51.369310 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:51.369274    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"]
Sep 14 03:25:51.369702 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:51.369486    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t" podUID="0d18248d-5877-4afd-881a-21f6441993e1" containerName="istio-proxy" containerID="cri-o://2d422940da295c1e69909277e975787034c221f3b72fd547dfa7c5948d31661e" gracePeriod=30
Sep 14 03:25:56.612167 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.612144    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:25:56.737003 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.736934    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-workload-certs\") pod \"0d18248d-5877-4afd-881a-21f6441993e1\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") "
Sep 14 03:25:56.737003 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.736969    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d18248d-5877-4afd-881a-21f6441993e1-istiod-ca-cert\") pod \"0d18248d-5877-4afd-881a-21f6441993e1\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") "
Sep 14 03:25:56.737003 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.736993    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbvvn\" (UniqueName: \"kubernetes.io/projected/0d18248d-5877-4afd-881a-21f6441993e1-kube-api-access-rbvvn\") pod \"0d18248d-5877-4afd-881a-21f6441993e1\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") "
Sep 14 03:25:56.737269 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.737026    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-workload-socket\") pod \"0d18248d-5877-4afd-881a-21f6441993e1\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") "
Sep 14 03:25:56.737269 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.737054    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d18248d-5877-4afd-881a-21f6441993e1-istio-podinfo\") pod \"0d18248d-5877-4afd-881a-21f6441993e1\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") "
Sep 14 03:25:56.737269 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.737131    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d18248d-5877-4afd-881a-21f6441993e1-istio-token\") pod \"0d18248d-5877-4afd-881a-21f6441993e1\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") "
Sep 14 03:25:56.737269 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.737157    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-credential-socket\") pod \"0d18248d-5877-4afd-881a-21f6441993e1\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") "
Sep 14 03:25:56.737269 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.737220    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-istio-envoy\") pod \"0d18248d-5877-4afd-881a-21f6441993e1\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") "
Sep 14 03:25:56.737269 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.737217    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0d18248d-5877-4afd-881a-21f6441993e1" (UID: "0d18248d-5877-4afd-881a-21f6441993e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:25:56.737592 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.737254    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-istio-data\") pod \"0d18248d-5877-4afd-881a-21f6441993e1\" (UID: \"0d18248d-5877-4afd-881a-21f6441993e1\") "
Sep 14 03:25:56.737592 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.737345    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0d18248d-5877-4afd-881a-21f6441993e1" (UID: "0d18248d-5877-4afd-881a-21f6441993e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:25:56.737592 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.737367    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d18248d-5877-4afd-881a-21f6441993e1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0d18248d-5877-4afd-881a-21f6441993e1" (UID: "0d18248d-5877-4afd-881a-21f6441993e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:25:56.737592 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.737464    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-istio-data" (OuterVolumeSpecName: "istio-data") pod "0d18248d-5877-4afd-881a-21f6441993e1" (UID: "0d18248d-5877-4afd-881a-21f6441993e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:25:56.737592 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.737478    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0d18248d-5877-4afd-881a-21f6441993e1" (UID: "0d18248d-5877-4afd-881a-21f6441993e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:25:56.737830 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.737670    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-workload-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:56.737830 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.737810    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-credential-socket\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:56.737830 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.737826    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-istio-data\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:56.737978 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.737839    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-workload-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:56.737978 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.737858    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d18248d-5877-4afd-881a-21f6441993e1-istiod-ca-cert\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:56.739197 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.739148    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d18248d-5877-4afd-881a-21f6441993e1-kube-api-access-rbvvn" (OuterVolumeSpecName: "kube-api-access-rbvvn") pod "0d18248d-5877-4afd-881a-21f6441993e1" (UID: "0d18248d-5877-4afd-881a-21f6441993e1"). InnerVolumeSpecName "kube-api-access-rbvvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:25:56.739335 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.739316    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d18248d-5877-4afd-881a-21f6441993e1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0d18248d-5877-4afd-881a-21f6441993e1" (UID: "0d18248d-5877-4afd-881a-21f6441993e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 03:25:56.739913 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.739893    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d18248d-5877-4afd-881a-21f6441993e1-istio-token" (OuterVolumeSpecName: "istio-token") pod "0d18248d-5877-4afd-881a-21f6441993e1" (UID: "0d18248d-5877-4afd-881a-21f6441993e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:25:56.739913 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.739895    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0d18248d-5877-4afd-881a-21f6441993e1" (UID: "0d18248d-5877-4afd-881a-21f6441993e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:25:56.838578 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.838551    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbvvn\" (UniqueName: \"kubernetes.io/projected/0d18248d-5877-4afd-881a-21f6441993e1-kube-api-access-rbvvn\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:56.838578 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.838574    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d18248d-5877-4afd-881a-21f6441993e1-istio-podinfo\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:56.838578 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.838583    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d18248d-5877-4afd-881a-21f6441993e1-istio-token\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:56.838761 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:56.838593    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d18248d-5877-4afd-881a-21f6441993e1-istio-envoy\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:25:57.234287 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:57.234252    2579 generic.go:356] "Generic (PLEG): container finished" podID="0d18248d-5877-4afd-881a-21f6441993e1" containerID="2d422940da295c1e69909277e975787034c221f3b72fd547dfa7c5948d31661e" exitCode=0
Sep 14 03:25:57.234459 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:57.234308    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"
Sep 14 03:25:57.234459 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:57.234304    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t" event={"ID":"0d18248d-5877-4afd-881a-21f6441993e1","Type":"ContainerDied","Data":"2d422940da295c1e69909277e975787034c221f3b72fd547dfa7c5948d31661e"}
Sep 14 03:25:57.234459 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:57.234377    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t" event={"ID":"0d18248d-5877-4afd-881a-21f6441993e1","Type":"ContainerDied","Data":"50a5a5f9885270c0319171005bbe2e81bbb72942a71db9e53f656f7b4fc688a0"}
Sep 14 03:25:57.234459 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:57.234402    2579 scope.go:117] "RemoveContainer" containerID="2d422940da295c1e69909277e975787034c221f3b72fd547dfa7c5948d31661e"
Sep 14 03:25:57.246688 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:57.246654    2579 scope.go:117] "RemoveContainer" containerID="9440856ddb3a1a3b4eba241ff4f696a53c3930921561e03392f80f729d2d1c0e"
Sep 14 03:25:57.255115 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:57.255009    2579 scope.go:117] "RemoveContainer" containerID="2d422940da295c1e69909277e975787034c221f3b72fd547dfa7c5948d31661e"
Sep 14 03:25:57.255705 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:25:57.255677    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d422940da295c1e69909277e975787034c221f3b72fd547dfa7c5948d31661e\": container with ID starting with 2d422940da295c1e69909277e975787034c221f3b72fd547dfa7c5948d31661e not found: ID does not exist" containerID="2d422940da295c1e69909277e975787034c221f3b72fd547dfa7c5948d31661e"
Sep 14 03:25:57.255857 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:57.255724    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d422940da295c1e69909277e975787034c221f3b72fd547dfa7c5948d31661e"} err="failed to get container status \"2d422940da295c1e69909277e975787034c221f3b72fd547dfa7c5948d31661e\": rpc error: code = NotFound desc = could not find container \"2d422940da295c1e69909277e975787034c221f3b72fd547dfa7c5948d31661e\": container with ID starting with 2d422940da295c1e69909277e975787034c221f3b72fd547dfa7c5948d31661e not found: ID does not exist"
Sep 14 03:25:57.255857 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:57.255747    2579 scope.go:117] "RemoveContainer" containerID="9440856ddb3a1a3b4eba241ff4f696a53c3930921561e03392f80f729d2d1c0e"
Sep 14 03:25:57.256125 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:25:57.256067    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9440856ddb3a1a3b4eba241ff4f696a53c3930921561e03392f80f729d2d1c0e\": container with ID starting with 9440856ddb3a1a3b4eba241ff4f696a53c3930921561e03392f80f729d2d1c0e not found: ID does not exist" containerID="9440856ddb3a1a3b4eba241ff4f696a53c3930921561e03392f80f729d2d1c0e"
Sep 14 03:25:57.256125 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:57.256101    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9440856ddb3a1a3b4eba241ff4f696a53c3930921561e03392f80f729d2d1c0e"} err="failed to get container status \"9440856ddb3a1a3b4eba241ff4f696a53c3930921561e03392f80f729d2d1c0e\": rpc error: code = NotFound desc = could not find container \"9440856ddb3a1a3b4eba241ff4f696a53c3930921561e03392f80f729d2d1c0e\": container with ID starting with 9440856ddb3a1a3b4eba241ff4f696a53c3930921561e03392f80f729d2d1c0e not found: ID does not exist"
Sep 14 03:25:57.257874 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:57.257855    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"]
Sep 14 03:25:57.261547 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:57.261528    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-177593-openshift-default-75f8f59f54-z6x5t"]
Sep 14 03:25:57.743625 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:25:57.743596    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d18248d-5877-4afd-881a-21f6441993e1" path="/var/lib/kubelet/pods/0d18248d-5877-4afd-881a-21f6441993e1/volumes"
Sep 14 03:26:18.197087 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:26:18.197055    2579 scope.go:117] "RemoveContainer" containerID="0867d47a0a7ad5098202c0f80f2c477c40bf5e3bf567da91e56f01d16566f135"
Sep 14 03:28:31.097515 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:31.097479    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"]
Sep 14 03:28:31.100151 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:31.097691    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm" podUID="eaf38a4b-1bf6-43c3-9c09-489a472f1fd2" containerName="manager" containerID="cri-o://88fa8d06dec43408d48ad37b302782aa09d56ab0044ba0bf0d2a8889b205df66" gracePeriod=10
Sep 14 03:28:33.978129 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:33.978096    2579 generic.go:356] "Generic (PLEG): container finished" podID="eaf38a4b-1bf6-43c3-9c09-489a472f1fd2" containerID="88fa8d06dec43408d48ad37b302782aa09d56ab0044ba0bf0d2a8889b205df66" exitCode=0
Sep 14 03:28:33.978482 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:33.978168    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm" event={"ID":"eaf38a4b-1bf6-43c3-9c09-489a472f1fd2","Type":"ContainerDied","Data":"88fa8d06dec43408d48ad37b302782aa09d56ab0044ba0bf0d2a8889b205df66"}
Sep 14 03:28:34.052864 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:34.052841    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"
Sep 14 03:28:34.105382 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:34.105351    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6nmj\" (UniqueName: \"kubernetes.io/projected/eaf38a4b-1bf6-43c3-9c09-489a472f1fd2-kube-api-access-k6nmj\") pod \"eaf38a4b-1bf6-43c3-9c09-489a472f1fd2\" (UID: \"eaf38a4b-1bf6-43c3-9c09-489a472f1fd2\") "
Sep 14 03:28:34.105539 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:34.105406    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eaf38a4b-1bf6-43c3-9c09-489a472f1fd2-extensions-socket-volume\") pod \"eaf38a4b-1bf6-43c3-9c09-489a472f1fd2\" (UID: \"eaf38a4b-1bf6-43c3-9c09-489a472f1fd2\") "
Sep 14 03:28:34.105812 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:34.105790    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaf38a4b-1bf6-43c3-9c09-489a472f1fd2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "eaf38a4b-1bf6-43c3-9c09-489a472f1fd2" (UID: "eaf38a4b-1bf6-43c3-9c09-489a472f1fd2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 03:28:34.107638 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:34.107614    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaf38a4b-1bf6-43c3-9c09-489a472f1fd2-kube-api-access-k6nmj" (OuterVolumeSpecName: "kube-api-access-k6nmj") pod "eaf38a4b-1bf6-43c3-9c09-489a472f1fd2" (UID: "eaf38a4b-1bf6-43c3-9c09-489a472f1fd2"). InnerVolumeSpecName "kube-api-access-k6nmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:28:34.206430 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:34.206354    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6nmj\" (UniqueName: \"kubernetes.io/projected/eaf38a4b-1bf6-43c3-9c09-489a472f1fd2-kube-api-access-k6nmj\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:28:34.206430 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:34.206385    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eaf38a4b-1bf6-43c3-9c09-489a472f1fd2-extensions-socket-volume\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:28:34.983391 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:34.983365    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"
Sep 14 03:28:34.983790 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:34.983401    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm" event={"ID":"eaf38a4b-1bf6-43c3-9c09-489a472f1fd2","Type":"ContainerDied","Data":"f8f1d4370d8de3e1846cfd0ad6a98054d037b7d3582f03f310b51361f6826c0f"}
Sep 14 03:28:34.983790 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:34.983462    2579 scope.go:117] "RemoveContainer" containerID="88fa8d06dec43408d48ad37b302782aa09d56ab0044ba0bf0d2a8889b205df66"
Sep 14 03:28:35.006398 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:35.006372    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"]
Sep 14 03:28:35.011064 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:35.011039    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-44gtm"]
Sep 14 03:28:35.742936 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:28:35.742906    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eaf38a4b-1bf6-43c3-9c09-489a472f1fd2" path="/var/lib/kubelet/pods/eaf38a4b-1bf6-43c3-9c09-489a472f1fd2/volumes"
Sep 14 03:29:31.789303 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789270    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-pqbqt"]
Sep 14 03:29:31.789701 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789589    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaf38a4b-1bf6-43c3-9c09-489a472f1fd2" containerName="manager"
Sep 14 03:29:31.789701 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789600    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaf38a4b-1bf6-43c3-9c09-489a472f1fd2" containerName="manager"
Sep 14 03:29:31.789701 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789611    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5da2703a-da04-4553-ad09-b76929116540" containerName="istio-proxy"
Sep 14 03:29:31.789701 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789617    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5da2703a-da04-4553-ad09-b76929116540" containerName="istio-proxy"
Sep 14 03:29:31.789701 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789623    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5da2703a-da04-4553-ad09-b76929116540" containerName="istio-proxy"
Sep 14 03:29:31.789701 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789628    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5da2703a-da04-4553-ad09-b76929116540" containerName="istio-proxy"
Sep 14 03:29:31.789701 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789641    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d18248d-5877-4afd-881a-21f6441993e1" containerName="istio-proxy"
Sep 14 03:29:31.789701 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789646    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d18248d-5877-4afd-881a-21f6441993e1" containerName="istio-proxy"
Sep 14 03:29:31.789701 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789652    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d18248d-5877-4afd-881a-21f6441993e1" containerName="istio-proxy"
Sep 14 03:29:31.789701 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789657    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d18248d-5877-4afd-881a-21f6441993e1" containerName="istio-proxy"
Sep 14 03:29:31.789701 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789666    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="575c0e79-4bd5-4f3b-af3a-f4f62212745f" containerName="revoke-keys"
Sep 14 03:29:31.789701 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789671    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="575c0e79-4bd5-4f3b-af3a-f4f62212745f" containerName="revoke-keys"
Sep 14 03:29:31.790101 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789727    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d18248d-5877-4afd-881a-21f6441993e1" containerName="istio-proxy"
Sep 14 03:29:31.790101 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789735    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="575c0e79-4bd5-4f3b-af3a-f4f62212745f" containerName="revoke-keys"
Sep 14 03:29:31.790101 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789743    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaf38a4b-1bf6-43c3-9c09-489a472f1fd2" containerName="manager"
Sep 14 03:29:31.790101 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789748    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5da2703a-da04-4553-ad09-b76929116540" containerName="istio-proxy"
Sep 14 03:29:31.790101 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.789754    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5da2703a-da04-4553-ad09-b76929116540" containerName="istio-proxy"
Sep 14 03:29:31.792945 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.792928    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-pqbqt"
Sep 14 03:29:31.795865 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.795844    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8sprk\""
Sep 14 03:29:31.806045 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.806023    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-pqbqt"]
Sep 14 03:29:31.858373 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.858348    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/97fb1056-296b-4f4a-aa83-5c41e4663670-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-pqbqt\" (UID: \"97fb1056-296b-4f4a-aa83-5c41e4663670\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-pqbqt"
Sep 14 03:29:31.858498 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.858376    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csn5f\" (UniqueName: \"kubernetes.io/projected/97fb1056-296b-4f4a-aa83-5c41e4663670-kube-api-access-csn5f\") pod \"kuadrant-operator-controller-manager-5769f99d9f-pqbqt\" (UID: \"97fb1056-296b-4f4a-aa83-5c41e4663670\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-pqbqt"
Sep 14 03:29:31.959749 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.959698    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/97fb1056-296b-4f4a-aa83-5c41e4663670-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-pqbqt\" (UID: \"97fb1056-296b-4f4a-aa83-5c41e4663670\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-pqbqt"
Sep 14 03:29:31.959749 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.959752    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csn5f\" (UniqueName: \"kubernetes.io/projected/97fb1056-296b-4f4a-aa83-5c41e4663670-kube-api-access-csn5f\") pod \"kuadrant-operator-controller-manager-5769f99d9f-pqbqt\" (UID: \"97fb1056-296b-4f4a-aa83-5c41e4663670\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-pqbqt"
Sep 14 03:29:31.960083 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.960064    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/97fb1056-296b-4f4a-aa83-5c41e4663670-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-pqbqt\" (UID: \"97fb1056-296b-4f4a-aa83-5c41e4663670\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-pqbqt"
Sep 14 03:29:31.990104 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:31.990074    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csn5f\" (UniqueName: \"kubernetes.io/projected/97fb1056-296b-4f4a-aa83-5c41e4663670-kube-api-access-csn5f\") pod \"kuadrant-operator-controller-manager-5769f99d9f-pqbqt\" (UID: \"97fb1056-296b-4f4a-aa83-5c41e4663670\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-pqbqt"
Sep 14 03:29:32.103106 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:32.103042    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-pqbqt"
Sep 14 03:29:32.382516 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:32.382461    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-pqbqt"]
Sep 14 03:29:32.384943 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:29:32.384901    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97fb1056_296b_4f4a_aa83_5c41e4663670.slice/crio-24b3f9419a1d159c3c6a6800ecfe52942f268386097c5f9ee0413c1a318b1db9 WatchSource:0}: Error finding container 24b3f9419a1d159c3c6a6800ecfe52942f268386097c5f9ee0413c1a318b1db9: Status 404 returned error can't find the container with id 24b3f9419a1d159c3c6a6800ecfe52942f268386097c5f9ee0413c1a318b1db9
Sep 14 03:29:33.256289 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:33.256253    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-pqbqt" event={"ID":"97fb1056-296b-4f4a-aa83-5c41e4663670","Type":"ContainerStarted","Data":"db43d8024577bb5af927e5b55abc10681b41cf9a538930caea5552a18db02c1f"}
Sep 14 03:29:33.256673 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:33.256317    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-pqbqt" event={"ID":"97fb1056-296b-4f4a-aa83-5c41e4663670","Type":"ContainerStarted","Data":"24b3f9419a1d159c3c6a6800ecfe52942f268386097c5f9ee0413c1a318b1db9"}
Sep 14 03:29:33.256673 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:33.256345    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-pqbqt"
Sep 14 03:29:33.290976 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:33.290918    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-pqbqt" podStartSLOduration=2.290901947 podStartE2EDuration="2.290901947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:29:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:29:33.289399875 +0000 UTC m=+2236.075461787" watchObservedRunningTime="2026-09-14 03:29:33.290901947 +0000 UTC m=+2236.076963859"
Sep 14 03:29:44.264111 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:29:44.264036    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-pqbqt"
Sep 14 03:30:00.139849 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:30:00.139820    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822610-zlsjd"]
Sep 14 03:30:00.140295 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:30:00.140213    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d18248d-5877-4afd-881a-21f6441993e1" containerName="istio-proxy"
Sep 14 03:30:00.142850 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:30:00.142835    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822610-zlsjd"
Sep 14 03:30:00.145673 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:30:00.145654    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sv5sd\""
Sep 14 03:30:00.151055 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:30:00.151023    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822610-zlsjd"]
Sep 14 03:30:00.159711 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:30:00.159691    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822610-zlsjd"
Sep 14 03:30:00.429888 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:30:00.429864    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822610-zlsjd"]
Sep 14 03:30:00.432126 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:30:00.432100    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2b8ade8_e271_41d3_81c7_0fe766a4162d.slice/crio-ad30626eee1fd1f10020789998aff72afaea6c6a5f0452b82afcc0c5ab12c0f6 WatchSource:0}: Error finding container ad30626eee1fd1f10020789998aff72afaea6c6a5f0452b82afcc0c5ab12c0f6: Status 404 returned error can't find the container with id ad30626eee1fd1f10020789998aff72afaea6c6a5f0452b82afcc0c5ab12c0f6
Sep 14 03:30:01.388344 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:30:01.388310    2579 generic.go:356] "Generic (PLEG): container finished" podID="c2b8ade8-e271-41d3-81c7-0fe766a4162d" containerID="f75e8ac85164f7ea54701ef72e25e5cc0e9276f7e8748c8bc7ecdda7c197dd3f" exitCode=0
Sep 14 03:30:01.388764 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:30:01.388398    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822610-zlsjd" event={"ID":"c2b8ade8-e271-41d3-81c7-0fe766a4162d","Type":"ContainerDied","Data":"f75e8ac85164f7ea54701ef72e25e5cc0e9276f7e8748c8bc7ecdda7c197dd3f"}
Sep 14 03:30:01.388764 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:30:01.388448    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822610-zlsjd" event={"ID":"c2b8ade8-e271-41d3-81c7-0fe766a4162d","Type":"ContainerStarted","Data":"ad30626eee1fd1f10020789998aff72afaea6c6a5f0452b82afcc0c5ab12c0f6"}
Sep 14 03:30:02.519788 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:30:02.519768    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822610-zlsjd"
Sep 14 03:30:03.399869 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:30:03.399841    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822610-zlsjd"
Sep 14 03:30:03.400034 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:30:03.399838    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822610-zlsjd" event={"ID":"c2b8ade8-e271-41d3-81c7-0fe766a4162d","Type":"ContainerDied","Data":"ad30626eee1fd1f10020789998aff72afaea6c6a5f0452b82afcc0c5ab12c0f6"}
Sep 14 03:30:03.400034 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:30:03.399963    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad30626eee1fd1f10020789998aff72afaea6c6a5f0452b82afcc0c5ab12c0f6"
Sep 14 03:30:18.254133 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:30:18.254099    2579 scope.go:117] "RemoveContainer" containerID="6a75105428ac5a27523ca6c9209fd961b064ae7cb773e9af994ca7b09fca4f15"
Sep 14 03:31:18.276108 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:18.276031    2579 scope.go:117] "RemoveContainer" containerID="9d2a2d7b830a3f607a81dee5f1b0e47cf4287d09fd4d2091705a61762876950a"
Sep 14 03:31:28.426631 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.426596    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"]
Sep 14 03:31:28.427021 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.426969    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2b8ade8-e271-41d3-81c7-0fe766a4162d" containerName="cleanup"
Sep 14 03:31:28.427021 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.426981    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2b8ade8-e271-41d3-81c7-0fe766a4162d" containerName="cleanup"
Sep 14 03:31:28.427099 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.427051    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2b8ade8-e271-41d3-81c7-0fe766a4162d" containerName="cleanup"
Sep 14 03:31:28.429935 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.429915    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:28.434024 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.433998    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 03:31:28.434024 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.434024    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jf78h\""
Sep 14 03:31:28.434273 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.434010    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 03:31:28.434273 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.434072    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 14 03:31:28.434273 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.434126    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 03:31:28.440116 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.440095    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"]
Sep 14 03:31:28.503004 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.502979    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45432c9-84ef-4a33-9f54-00be164e8fbe-openshift-service-ca\") pod \"maas-api-564fdcb8d5-mbm7x\" (UID: \"e45432c9-84ef-4a33-9f54-00be164e8fbe\") " pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:28.503151 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.503017    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45432c9-84ef-4a33-9f54-00be164e8fbe-maas-api-tls\") pod \"maas-api-564fdcb8d5-mbm7x\" (UID: \"e45432c9-84ef-4a33-9f54-00be164e8fbe\") " pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:28.503151 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.503036    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfgnc\" (UniqueName: \"kubernetes.io/projected/e45432c9-84ef-4a33-9f54-00be164e8fbe-kube-api-access-hfgnc\") pod \"maas-api-564fdcb8d5-mbm7x\" (UID: \"e45432c9-84ef-4a33-9f54-00be164e8fbe\") " pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:28.503151 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.503063    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45432c9-84ef-4a33-9f54-00be164e8fbe-metrics-certs\") pod \"maas-api-564fdcb8d5-mbm7x\" (UID: \"e45432c9-84ef-4a33-9f54-00be164e8fbe\") " pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:28.604355 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.604319    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45432c9-84ef-4a33-9f54-00be164e8fbe-metrics-certs\") pod \"maas-api-564fdcb8d5-mbm7x\" (UID: \"e45432c9-84ef-4a33-9f54-00be164e8fbe\") " pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:28.604549 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.604385    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45432c9-84ef-4a33-9f54-00be164e8fbe-openshift-service-ca\") pod \"maas-api-564fdcb8d5-mbm7x\" (UID: \"e45432c9-84ef-4a33-9f54-00be164e8fbe\") " pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:28.604549 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.604422    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45432c9-84ef-4a33-9f54-00be164e8fbe-maas-api-tls\") pod \"maas-api-564fdcb8d5-mbm7x\" (UID: \"e45432c9-84ef-4a33-9f54-00be164e8fbe\") " pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:28.604549 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.604438    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfgnc\" (UniqueName: \"kubernetes.io/projected/e45432c9-84ef-4a33-9f54-00be164e8fbe-kube-api-access-hfgnc\") pod \"maas-api-564fdcb8d5-mbm7x\" (UID: \"e45432c9-84ef-4a33-9f54-00be164e8fbe\") " pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:28.605122 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.605094    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45432c9-84ef-4a33-9f54-00be164e8fbe-openshift-service-ca\") pod \"maas-api-564fdcb8d5-mbm7x\" (UID: \"e45432c9-84ef-4a33-9f54-00be164e8fbe\") " pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:28.606884 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.606864    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45432c9-84ef-4a33-9f54-00be164e8fbe-metrics-certs\") pod \"maas-api-564fdcb8d5-mbm7x\" (UID: \"e45432c9-84ef-4a33-9f54-00be164e8fbe\") " pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:28.606976 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.606952    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45432c9-84ef-4a33-9f54-00be164e8fbe-maas-api-tls\") pod \"maas-api-564fdcb8d5-mbm7x\" (UID: \"e45432c9-84ef-4a33-9f54-00be164e8fbe\") " pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:28.613366 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.613345    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfgnc\" (UniqueName: \"kubernetes.io/projected/e45432c9-84ef-4a33-9f54-00be164e8fbe-kube-api-access-hfgnc\") pod \"maas-api-564fdcb8d5-mbm7x\" (UID: \"e45432c9-84ef-4a33-9f54-00be164e8fbe\") " pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:28.742161 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:28.742137    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:29.021998 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:29.021812    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"]
Sep 14 03:31:29.022458 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:29.022431    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 03:31:29.799404 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:29.799361    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x" event={"ID":"e45432c9-84ef-4a33-9f54-00be164e8fbe","Type":"ContainerStarted","Data":"637d86e5df9c00f1703ee1dd48ce231848b883922b93b0a0e8694ade44bb859f"}
Sep 14 03:31:29.799848 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:29.799424    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x" event={"ID":"e45432c9-84ef-4a33-9f54-00be164e8fbe","Type":"ContainerStarted","Data":"956452adca33da675a66da915b956487e67992e49ee3e92454f5b5de5ac69d0b"}
Sep 14 03:31:29.799848 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:29.799452    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:29.818288 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:29.818236    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x" podStartSLOduration=1.178912313 podStartE2EDuration="1.818217228s" totalImagesPullingTime="639.304915ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 03:31:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:31:29.816603474 +0000 UTC m=+2352.602665397" watchObservedRunningTime="2026-09-14 03:31:29.818217228 +0000 UTC m=+2352.604279142"
Sep 14 03:31:35.809080 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:35.809051    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:35.999313 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:35.999281    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"]
Sep 14 03:31:35.999523 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:35.999482    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x" podUID="e45432c9-84ef-4a33-9f54-00be164e8fbe" containerName="maas-api" containerID="cri-o://637d86e5df9c00f1703ee1dd48ce231848b883922b93b0a0e8694ade44bb859f" gracePeriod=30
Sep 14 03:31:36.238718 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.238695    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:36.363667 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.363632    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45432c9-84ef-4a33-9f54-00be164e8fbe-maas-api-tls\") pod \"e45432c9-84ef-4a33-9f54-00be164e8fbe\" (UID: \"e45432c9-84ef-4a33-9f54-00be164e8fbe\") "
Sep 14 03:31:36.363860 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.363674    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfgnc\" (UniqueName: \"kubernetes.io/projected/e45432c9-84ef-4a33-9f54-00be164e8fbe-kube-api-access-hfgnc\") pod \"e45432c9-84ef-4a33-9f54-00be164e8fbe\" (UID: \"e45432c9-84ef-4a33-9f54-00be164e8fbe\") "
Sep 14 03:31:36.363860 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.363701    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45432c9-84ef-4a33-9f54-00be164e8fbe-metrics-certs\") pod \"e45432c9-84ef-4a33-9f54-00be164e8fbe\" (UID: \"e45432c9-84ef-4a33-9f54-00be164e8fbe\") "
Sep 14 03:31:36.363860 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.363820    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45432c9-84ef-4a33-9f54-00be164e8fbe-openshift-service-ca\") pod \"e45432c9-84ef-4a33-9f54-00be164e8fbe\" (UID: \"e45432c9-84ef-4a33-9f54-00be164e8fbe\") "
Sep 14 03:31:36.364224 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.364168    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e45432c9-84ef-4a33-9f54-00be164e8fbe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e45432c9-84ef-4a33-9f54-00be164e8fbe" (UID: "e45432c9-84ef-4a33-9f54-00be164e8fbe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 03:31:36.366047 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.365976    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e45432c9-84ef-4a33-9f54-00be164e8fbe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e45432c9-84ef-4a33-9f54-00be164e8fbe" (UID: "e45432c9-84ef-4a33-9f54-00be164e8fbe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:31:36.366154 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.366058    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e45432c9-84ef-4a33-9f54-00be164e8fbe-kube-api-access-hfgnc" (OuterVolumeSpecName: "kube-api-access-hfgnc") pod "e45432c9-84ef-4a33-9f54-00be164e8fbe" (UID: "e45432c9-84ef-4a33-9f54-00be164e8fbe"). InnerVolumeSpecName "kube-api-access-hfgnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 03:31:36.366154 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.366060    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e45432c9-84ef-4a33-9f54-00be164e8fbe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e45432c9-84ef-4a33-9f54-00be164e8fbe" (UID: "e45432c9-84ef-4a33-9f54-00be164e8fbe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 03:31:36.464911 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.464883    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfgnc\" (UniqueName: \"kubernetes.io/projected/e45432c9-84ef-4a33-9f54-00be164e8fbe-kube-api-access-hfgnc\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:31:36.464911 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.464907    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45432c9-84ef-4a33-9f54-00be164e8fbe-metrics-certs\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:31:36.464911 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.464916    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45432c9-84ef-4a33-9f54-00be164e8fbe-openshift-service-ca\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:31:36.465092 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.464923    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45432c9-84ef-4a33-9f54-00be164e8fbe-maas-api-tls\") on node \"ip-10-0-138-244.ec2.internal\" DevicePath \"\""
Sep 14 03:31:36.830805 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.830777    2579 generic.go:356] "Generic (PLEG): container finished" podID="e45432c9-84ef-4a33-9f54-00be164e8fbe" containerID="637d86e5df9c00f1703ee1dd48ce231848b883922b93b0a0e8694ade44bb859f" exitCode=0
Sep 14 03:31:36.831225 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.830819    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"
Sep 14 03:31:36.831225 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.830847    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x" event={"ID":"e45432c9-84ef-4a33-9f54-00be164e8fbe","Type":"ContainerDied","Data":"637d86e5df9c00f1703ee1dd48ce231848b883922b93b0a0e8694ade44bb859f"}
Sep 14 03:31:36.831225 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.830888    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x" event={"ID":"e45432c9-84ef-4a33-9f54-00be164e8fbe","Type":"ContainerDied","Data":"956452adca33da675a66da915b956487e67992e49ee3e92454f5b5de5ac69d0b"}
Sep 14 03:31:36.831225 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.830911    2579 scope.go:117] "RemoveContainer" containerID="637d86e5df9c00f1703ee1dd48ce231848b883922b93b0a0e8694ade44bb859f"
Sep 14 03:31:36.840153 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.840137    2579 scope.go:117] "RemoveContainer" containerID="637d86e5df9c00f1703ee1dd48ce231848b883922b93b0a0e8694ade44bb859f"
Sep 14 03:31:36.840436 ip-10-0-138-244 kubenswrapper[2579]: E0914 03:31:36.840412    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"637d86e5df9c00f1703ee1dd48ce231848b883922b93b0a0e8694ade44bb859f\": container with ID starting with 637d86e5df9c00f1703ee1dd48ce231848b883922b93b0a0e8694ade44bb859f not found: ID does not exist" containerID="637d86e5df9c00f1703ee1dd48ce231848b883922b93b0a0e8694ade44bb859f"
Sep 14 03:31:36.840527 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.840442    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"637d86e5df9c00f1703ee1dd48ce231848b883922b93b0a0e8694ade44bb859f"} err="failed to get container status \"637d86e5df9c00f1703ee1dd48ce231848b883922b93b0a0e8694ade44bb859f\": rpc error: code = NotFound desc = could not find container \"637d86e5df9c00f1703ee1dd48ce231848b883922b93b0a0e8694ade44bb859f\": container with ID starting with 637d86e5df9c00f1703ee1dd48ce231848b883922b93b0a0e8694ade44bb859f not found: ID does not exist"
Sep 14 03:31:36.852021 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.851948    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"]
Sep 14 03:31:36.853477 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:36.853458    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-564fdcb8d5-mbm7x"]
Sep 14 03:31:37.743771 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:31:37.743740    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e45432c9-84ef-4a33-9f54-00be164e8fbe" path="/var/lib/kubelet/pods/e45432c9-84ef-4a33-9f54-00be164e8fbe/volumes"
Sep 14 03:32:02.119354 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:02.119301    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-252ng_8c76688b-dff6-4370-970e-68125a400067/manager/0.log"
Sep 14 03:32:02.578410 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:02.578386    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4724d_163a441e-6c6a-4a9c-a372-40d4d7050af3/server/0.log"
Sep 14 03:32:02.971136 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:02.971060    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67ffb7699c-85hz6_1a6bb5e4-d156-4977-9b35-249562731e42/manager/0.log"
Sep 14 03:32:02.985081 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:02.985061    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67ffb7699c-85hz6_1a6bb5e4-d156-4977-9b35-249562731e42/copy-manifests/0.log"
Sep 14 03:32:03.718332 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:03.718303    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29822595-8pmv6_9651dd66-4bad-4bb3-a8c3-d6d3bc5d540d/cleanup/0.log"
Sep 14 03:32:03.843317 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:03.843288    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29822610-zlsjd_c2b8ade8-e271-41d3-81c7-0fe766a4162d/cleanup/0.log"
Sep 14 03:32:04.900970 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:04.900937    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-vqfvm_4f2de835-c0c9-4820-b7c2-05a64373675e/manager/0.log"
Sep 14 03:32:05.299149 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:05.299121    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-pqbqt_97fb1056-296b-4f4a-aa83-5c41e4663670/manager/0.log"
Sep 14 03:32:06.104998 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:06.104970    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-9kzn4_c56b5cb9-d498-4d61-b120-c30d5faef3a9/discovery/0.log"
Sep 14 03:32:06.347539 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:06.347517    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-798674575b-dp5dl_a43bf689-dde6-4944-a08a-d012c421df06/kube-auth-proxy/0.log"
Sep 14 03:32:07.242089 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:07.242049    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-4fbrp_db52f177-a321-4cb3-88d0-05a91f888574/main/0.log"
Sep 14 03:32:07.377291 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:07.377264    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-zh8mk_ed62ad4e-025d-4589-aca5-2eb249f62739/main/0.log"
Sep 14 03:32:07.657211 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:07.657122    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-t92th_f800c2dd-db74-4a77-98db-95de661306b7/main/0.log"
Sep 14 03:32:17.535934 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:17.535900    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h5wrd_2407936d-8e27-4f88-93a8-9dca1a2b80b9/global-pull-secret-syncer/0.log"
Sep 14 03:32:17.789090 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:17.788997    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6wg97_f0a6b9f0-9453-482b-aa52-a8526bdfcf15/konnectivity-agent/0.log"
Sep 14 03:32:17.937180 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:17.937156    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-244.ec2.internal_be05b9bf67903a6091aac71677a6fb10/haproxy/0.log"
Sep 14 03:32:18.294253 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:18.294224    2579 scope.go:117] "RemoveContainer" containerID="20707cedfd7c55a94997901b190f33dcff728abe69ba7a543d349954736dfacf"
Sep 14 03:32:22.783403 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:22.783371    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-vqfvm_4f2de835-c0c9-4820-b7c2-05a64373675e/manager/0.log"
Sep 14 03:32:23.032735 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:23.032709    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-pqbqt_97fb1056-296b-4f4a-aa83-5c41e4663670/manager/0.log"
Sep 14 03:32:25.454810 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:25.454785    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-88lxg_4e5107dd-669d-4e75-8a27-63b58d6fd6b5/node-exporter/0.log"
Sep 14 03:32:25.503973 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:25.503949    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-88lxg_4e5107dd-669d-4e75-8a27-63b58d6fd6b5/kube-rbac-proxy/0.log"
Sep 14 03:32:25.556930 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:25.556906    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-88lxg_4e5107dd-669d-4e75-8a27-63b58d6fd6b5/init-textfile/0.log"
Sep 14 03:32:26.304533 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:26.304508    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-mc26m_40fdddc8-0da1-4037-8c59-8b616f5ee781/prometheus-operator-admission-webhook/0.log"
Sep 14 03:32:27.058021 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.057989    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"]
Sep 14 03:32:27.058420 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.058342    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e45432c9-84ef-4a33-9f54-00be164e8fbe" containerName="maas-api"
Sep 14 03:32:27.058420 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.058353    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e45432c9-84ef-4a33-9f54-00be164e8fbe" containerName="maas-api"
Sep 14 03:32:27.058420 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.058413    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e45432c9-84ef-4a33-9f54-00be164e8fbe" containerName="maas-api"
Sep 14 03:32:27.061314 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.061297    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.068627 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.068605    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-f2kzn\"/\"default-dockercfg-v4n8b\""
Sep 14 03:32:27.069290 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.069271    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f2kzn\"/\"kube-root-ca.crt\""
Sep 14 03:32:27.069414 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.069317    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f2kzn\"/\"openshift-service-ca.crt\""
Sep 14 03:32:27.080706 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.080679    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"]
Sep 14 03:32:27.184011 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.183980    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2009aa4a-2dbf-409e-a11d-23ab12f0f572-lib-modules\") pod \"perf-node-gather-daemonset-mr58k\" (UID: \"2009aa4a-2dbf-409e-a11d-23ab12f0f572\") " pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.184011 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.184010    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2009aa4a-2dbf-409e-a11d-23ab12f0f572-proc\") pod \"perf-node-gather-daemonset-mr58k\" (UID: \"2009aa4a-2dbf-409e-a11d-23ab12f0f572\") " pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.184227 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.184047    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2009aa4a-2dbf-409e-a11d-23ab12f0f572-podres\") pod \"perf-node-gather-daemonset-mr58k\" (UID: \"2009aa4a-2dbf-409e-a11d-23ab12f0f572\") " pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.184227 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.184104    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl7x9\" (UniqueName: \"kubernetes.io/projected/2009aa4a-2dbf-409e-a11d-23ab12f0f572-kube-api-access-tl7x9\") pod \"perf-node-gather-daemonset-mr58k\" (UID: \"2009aa4a-2dbf-409e-a11d-23ab12f0f572\") " pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.184227 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.184159    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2009aa4a-2dbf-409e-a11d-23ab12f0f572-sys\") pod \"perf-node-gather-daemonset-mr58k\" (UID: \"2009aa4a-2dbf-409e-a11d-23ab12f0f572\") " pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.284734 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.284702    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2009aa4a-2dbf-409e-a11d-23ab12f0f572-podres\") pod \"perf-node-gather-daemonset-mr58k\" (UID: \"2009aa4a-2dbf-409e-a11d-23ab12f0f572\") " pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.284886 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.284745    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl7x9\" (UniqueName: \"kubernetes.io/projected/2009aa4a-2dbf-409e-a11d-23ab12f0f572-kube-api-access-tl7x9\") pod \"perf-node-gather-daemonset-mr58k\" (UID: \"2009aa4a-2dbf-409e-a11d-23ab12f0f572\") " pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.284886 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.284781    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2009aa4a-2dbf-409e-a11d-23ab12f0f572-sys\") pod \"perf-node-gather-daemonset-mr58k\" (UID: \"2009aa4a-2dbf-409e-a11d-23ab12f0f572\") " pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.284886 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.284842    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2009aa4a-2dbf-409e-a11d-23ab12f0f572-lib-modules\") pod \"perf-node-gather-daemonset-mr58k\" (UID: \"2009aa4a-2dbf-409e-a11d-23ab12f0f572\") " pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.284886 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.284855    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2009aa4a-2dbf-409e-a11d-23ab12f0f572-podres\") pod \"perf-node-gather-daemonset-mr58k\" (UID: \"2009aa4a-2dbf-409e-a11d-23ab12f0f572\") " pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.284886 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.284865    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2009aa4a-2dbf-409e-a11d-23ab12f0f572-proc\") pod \"perf-node-gather-daemonset-mr58k\" (UID: \"2009aa4a-2dbf-409e-a11d-23ab12f0f572\") " pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.285087 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.284909    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2009aa4a-2dbf-409e-a11d-23ab12f0f572-sys\") pod \"perf-node-gather-daemonset-mr58k\" (UID: \"2009aa4a-2dbf-409e-a11d-23ab12f0f572\") " pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.285087 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.284919    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2009aa4a-2dbf-409e-a11d-23ab12f0f572-proc\") pod \"perf-node-gather-daemonset-mr58k\" (UID: \"2009aa4a-2dbf-409e-a11d-23ab12f0f572\") " pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.285087 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.284977    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2009aa4a-2dbf-409e-a11d-23ab12f0f572-lib-modules\") pod \"perf-node-gather-daemonset-mr58k\" (UID: \"2009aa4a-2dbf-409e-a11d-23ab12f0f572\") " pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.295340 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.295316    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl7x9\" (UniqueName: \"kubernetes.io/projected/2009aa4a-2dbf-409e-a11d-23ab12f0f572-kube-api-access-tl7x9\") pod \"perf-node-gather-daemonset-mr58k\" (UID: \"2009aa4a-2dbf-409e-a11d-23ab12f0f572\") " pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.377447 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.377382    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:27.653049 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.653026    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"]
Sep 14 03:32:27.654899 ip-10-0-138-244 kubenswrapper[2579]: W0914 03:32:27.654873    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2009aa4a_2dbf_409e_a11d_23ab12f0f572.slice/crio-2aba211ba00c4cde58b239c2e28657acffcee06f24b66ec1e0abdd9c8e2b4c52 WatchSource:0}: Error finding container 2aba211ba00c4cde58b239c2e28657acffcee06f24b66ec1e0abdd9c8e2b4c52: Status 404 returned error can't find the container with id 2aba211ba00c4cde58b239c2e28657acffcee06f24b66ec1e0abdd9c8e2b4c52
Sep 14 03:32:27.920803 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:27.920726    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-2pxbm_b7851e60-80bc-4bdc-871c-acd2d69548b5/networking-console-plugin/0.log"
Sep 14 03:32:28.076936 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:28.076904    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k" event={"ID":"2009aa4a-2dbf-409e-a11d-23ab12f0f572","Type":"ContainerStarted","Data":"3339be69a9cf88f3b11a5ab500f6968b854a445430b5c6992e9b24c34fcfde4e"}
Sep 14 03:32:28.077354 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:28.076953    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k" event={"ID":"2009aa4a-2dbf-409e-a11d-23ab12f0f572","Type":"ContainerStarted","Data":"2aba211ba00c4cde58b239c2e28657acffcee06f24b66ec1e0abdd9c8e2b4c52"}
Sep 14 03:32:28.077354 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:28.077000    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:28.100926 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:28.100880    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k" podStartSLOduration=1.100861898 podStartE2EDuration="1.100861898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 03:32:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 03:32:28.098857177 +0000 UTC m=+2410.884919090" watchObservedRunningTime="2026-09-14 03:32:28.100861898 +0000 UTC m=+2410.886923812"
Sep 14 03:32:31.145747 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:31.145717    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pq276_56e1da36-b3ce-4cf4-b99c-dd9a7822c73c/dns/0.log"
Sep 14 03:32:31.175270 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:31.175247    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pq276_56e1da36-b3ce-4cf4-b99c-dd9a7822c73c/kube-rbac-proxy/0.log"
Sep 14 03:32:31.282434 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:31.282408    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mhwbd_1a65bc4c-0400-4b9f-ab60-75b0ff402476/dns-node-resolver/0.log"
Sep 14 03:32:31.964636 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:31.964600    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-57bff6854f-2wdw2_7f0eb029-15e9-43c1-82ca-19ac255e4c5b/registry/0.log"
Sep 14 03:32:32.057132 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:32.057105    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gp9kv_cae07d88-4fe4-41c8-b2f9-2b8abb3bc353/node-ca/0.log"
Sep 14 03:32:33.645064 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:33.645030    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-9kzn4_c56b5cb9-d498-4d61-b120-c30d5faef3a9/discovery/0.log"
Sep 14 03:32:33.729495 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:33.729470    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-798674575b-dp5dl_a43bf689-dde6-4944-a08a-d012c421df06/kube-auth-proxy/0.log"
Sep 14 03:32:34.092020 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:34.091996    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-f2kzn/perf-node-gather-daemonset-mr58k"
Sep 14 03:32:34.979059 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:34.979035    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-269h2_ea8c0670-0161-4347-ad46-98885e8edd2b/serve-healthcheck-canary/0.log"
Sep 14 03:32:35.688314 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:35.688281    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5zdn2_a890518d-c1d1-4ca7-bb6e-45032e701414/kube-rbac-proxy/0.log"
Sep 14 03:32:35.729407 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:35.729379    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5zdn2_a890518d-c1d1-4ca7-bb6e-45032e701414/exporter/0.log"
Sep 14 03:32:35.766986 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:35.766966    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5zdn2_a890518d-c1d1-4ca7-bb6e-45032e701414/extractor/0.log"
Sep 14 03:32:37.664964 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:37.664885    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-9kzn4_c56b5cb9-d498-4d61-b120-c30d5faef3a9/discovery/0.log"
Sep 14 03:32:37.717277 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:37.717257    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-798674575b-dp5dl_a43bf689-dde6-4944-a08a-d012c421df06/kube-auth-proxy/0.log"
Sep 14 03:32:38.606076 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:38.606012    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-252ng_8c76688b-dff6-4370-970e-68125a400067/manager/0.log"
Sep 14 03:32:39.063723 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:39.063697    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4724d_163a441e-6c6a-4a9c-a372-40d4d7050af3/server/0.log"
Sep 14 03:32:39.307378 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:39.307347    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67ffb7699c-85hz6_1a6bb5e4-d156-4977-9b35-249562731e42/manager/0.log"
Sep 14 03:32:39.339646 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:39.339586    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67ffb7699c-85hz6_1a6bb5e4-d156-4977-9b35-249562731e42/copy-manifests/0.log"
Sep 14 03:32:39.992950 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:39.992912    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-9kzn4_c56b5cb9-d498-4d61-b120-c30d5faef3a9/discovery/0.log"
Sep 14 03:32:40.031964 ip-10-0-138-244 kubenswrapper[2579]: I0914 03:32:40.031932    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-798674575b-dp5dl_a43bf689-dde6-4944-a08a-d012c421df06/kube-auth-proxy/0.log"