Jul 28 19:00:22.839626 ip-10-0-137-70 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 19:00:22.839837 ip-10-0-137-70 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 19:00:22.839933 ip-10-0-137-70 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 19:00:22.840395 ip-10-0-137-70 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 19:00:32.847273 ip-10-0-137-70 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 19:00:32.847294 ip-10-0-137-70 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot a84a526fc02d43d4953882b40600ec70 --
Jul 28 19:02:31.951855 ip-10-0-137-70 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 19:02:32.297630 ip-10-0-137-70 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:02:32.297630 ip-10-0-137-70 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 19:02:32.297630 ip-10-0-137-70 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:02:32.297630 ip-10-0-137-70 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 19:02:32.297630 ip-10-0-137-70 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:02:32.299960 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.299854    2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 19:02:32.305855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305829    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:32.305855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305854    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:32.305855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305858    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305861    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305865    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305868    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305871    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305874    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305878    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305881    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305884    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305886    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305889    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305894    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305897    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305899    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305902    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305905    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305908    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305911    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305914    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:32.305963 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305917    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305921    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305924    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305926    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305929    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305932    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305934    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305937    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305940    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305942    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305945    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305948    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305950    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305953    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305955    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305961    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305964    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305968    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305972    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305977    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:32.306476 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305980    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305983    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305986    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305989    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305992    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305994    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.305997    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306000    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306002    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306005    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306008    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306011    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306013    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306016    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306019    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306022    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306026    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306030    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306034    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306037    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:32.306995 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306040    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306042    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306045    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306047    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306050    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306053    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306055    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306058    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306061    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306072    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306081    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306085    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306089    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306092    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306095    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306098    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306101    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306103    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306106    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306109    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:32.307770 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306112    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306114    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306117    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306120    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306123    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306126    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306128    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306131    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306134    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306136    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306139    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306142    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306145    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306147    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306150    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306153    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306155    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.306158    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307279    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:32.308345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307302    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307307    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307314    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307319    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307326    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307334    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307339    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307343    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307348    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307352    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307357    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307361    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307370    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307375    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307379    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307383    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307388    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307395    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307401    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:32.308824 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307406    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307411    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307415    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307420    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307425    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307433    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307437    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307441    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307446    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307450    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307455    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307459    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307464    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307469    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307474    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307478    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307498    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307502    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307512    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307517    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:32.309310 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307521    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307525    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307529    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307534    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307543    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307548    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307552    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307560    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307565    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307569    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307578    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307582    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307587    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307592    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307596    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307601    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307605    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307609    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307614    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:32.309833 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307619    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307623    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307627    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307637    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307641    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307646    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307652    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307657    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307664    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307668    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307673    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307678    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307684    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307689    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307693    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307703    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307708    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307712    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307716    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307721    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:32.310307 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307726    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307731    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307735    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307740    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307744    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307749    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307754    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307763    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307768    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307772    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307777    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307782    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307786    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307791    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307795    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307799    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307804    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307808    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307813    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:32.310836 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.307817    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308921    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308936    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308940    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308945    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308949    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308953    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308956    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308959    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308962    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308965    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308968    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308970    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308975    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308979    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308982    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308984    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308987    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308990    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:32.311421 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308993    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308995    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.308998    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309000    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309003    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309006    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309009    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309011    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309014    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309016    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309026    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309029    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309032    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309035    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309038    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309041    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309046    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309050    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309053    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309056    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:32.311938 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309059    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309062    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309065    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309067    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309070    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309072    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309075    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309078    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309081    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309083    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309086    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309088    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309091    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309094    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309096    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309099    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309101    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309104    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309106    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309109    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:32.312459 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309111    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309114    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309116    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309119    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309122    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309124    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309128    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309132    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309134    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309137    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309140    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309143    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309146    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309148    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309151    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309154    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309156    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309159    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309161    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309164    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:32.312961 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309167    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309169    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309172    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309175    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309177    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309180    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309183    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309185    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309188    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309190    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309193    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309196    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309198    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309201    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309204    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309206    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309209    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309212    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309215    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309218    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:32.313501 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.309220    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309303    2561 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309313    2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309322    2561 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309327    2561 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309332    2561 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309336    2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309341    2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309346    2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309349    2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309353    2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309356    2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309360    2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309364    2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309368    2561 flags.go:64] FLAG: --cgroup-root=""
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309371    2561 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309374    2561 flags.go:64] FLAG: --client-ca-file=""
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309379    2561 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309382    2561 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309387    2561 flags.go:64] FLAG: --cluster-domain=""
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309390    2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309393    2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309397    2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309400    2561 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 19:02:32.313977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309405    2561 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309408    2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309411    2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309415    2561 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309418    2561 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309421    2561 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309426    2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309430    2561 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309433    2561 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309438    2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309441    2561 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309444    2561 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309447    2561 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309450    2561 flags.go:64] FLAG: --enable-server="true"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309454    2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309459    2561 flags.go:64] FLAG: --event-burst="100"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309462    2561 flags.go:64] FLAG: --event-qps="50"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309465    2561 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309468    2561 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309472    2561 flags.go:64] FLAG: --eviction-hard=""
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309476    2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309494    2561 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309497    2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309500    2561 flags.go:64] FLAG: --eviction-soft=""
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309504    2561 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 19:02:32.314580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309507    2561 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309511    2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309514    2561 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309517    2561 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309520    2561 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309523    2561 flags.go:64] FLAG: --feature-gates=""
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309527    2561 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309531    2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309534    2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309537    2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309541    2561 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309544    2561 flags.go:64] FLAG: --help="false"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309547    2561 flags.go:64] FLAG: --hostname-override="ip-10-0-137-70.ec2.internal"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309550    2561 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309555    2561 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309559    2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309562    2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309566    2561 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309570    2561 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309573    2561 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309576    2561 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309580    2561 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309584    2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309587    2561 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 19:02:32.315200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309590    2561 flags.go:64] FLAG: --kube-reserved=""
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309594    2561 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309597    2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309601    2561 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309604    2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309607    2561 flags.go:64] FLAG: --lock-file=""
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309610    2561 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309613    2561 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309616    2561 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309627    2561 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309631    2561 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309634    2561 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309638    2561 flags.go:64] FLAG: --logging-format="text"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309641    2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309644    2561 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309647    2561 flags.go:64] FLAG: --manifest-url=""
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309650    2561 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309655    2561 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309658    2561 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309663    2561 flags.go:64] FLAG: --max-pods="110"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309666    2561 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309669    2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309672    2561 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309675    2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309679    2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 19:02:32.315804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309682    2561 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309685    2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309692    2561 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309696    2561 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309699    2561 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309702    2561 flags.go:64] FLAG: --pod-cidr=""
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309705    2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309712    2561 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309715    2561 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309718    2561 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309721    2561 flags.go:64] FLAG: --port="10250"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309724    2561 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309727    2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0342b1a45af4e3943"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309731    2561 flags.go:64] FLAG: --qos-reserved=""
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309734    2561 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309737    2561 flags.go:64] FLAG: --register-node="true"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309740    2561 flags.go:64] FLAG: --register-with-taints=""
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309744    2561 flags.go:64] FLAG: --registry-burst="10"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309749    2561 flags.go:64] FLAG: --registry-qps="5"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309753    2561 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309756    2561 flags.go:64] FLAG: --reserved-memory=""
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309760    2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309763    2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309766    2561 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309770    2561 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 19:02:32.316448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309773    2561 flags.go:64] FLAG: --runonce="false"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309776    2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309779    2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309783    2561 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309786    2561 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309789    2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309792    2561 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309796    2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309800    2561 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309803    2561 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309806    2561 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309809    2561 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309812    2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309816    2561 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309819    2561 flags.go:64] FLAG: --system-cgroups=""
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309822    2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309827    2561 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309830    2561 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309834    2561 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309838    2561 flags.go:64] FLAG: --tls-min-version=""
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309841    2561 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309845    2561 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309848    2561 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309851    2561 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309854    2561 flags.go:64] FLAG: --v="2"
Jul 28 19:02:32.317118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309859    2561 flags.go:64] FLAG: --version="false"
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309864    2561 flags.go:64] FLAG: --vmodule=""
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309868    2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.309872    2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310001    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310008    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310011    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310014    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310017    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310020    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310023    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310029    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310032    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310034    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310038    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310041    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310044    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310046    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310049    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310052    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:32.317756 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310055    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310058    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310061    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310072    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310077    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310080    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310083    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310086    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310089    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310092    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310095    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310098    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310101    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310103    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310109    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310113    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310116    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310119    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310121    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:32.318281 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310124    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310127    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310130    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310132    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310136    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310139    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310142    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310144    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310147    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310150    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310153    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310156    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310158    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310161    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310163    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310166    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310168    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310171    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310174    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310177    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:32.318785 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310179    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310182    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310184    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310187    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310190    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310193    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310195    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310200    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310204    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310206    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310209    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310211    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310216    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310219    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310222    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310225    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310229    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310232    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310234    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:32.319344 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310237    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310239    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310242    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310245    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310248    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310251    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310253    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310256    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310259    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310262    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310264    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310267    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310270    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310272    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310275    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310277    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310280    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310282    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310285    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310288    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:32.319850 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310292    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:32.320345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310295    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:32.320345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310299    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:32.320345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310301    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:32.320345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.310304    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:32.320345 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.310799    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:02:32.320345 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.317664    2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 19:02:32.320345 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.317692    2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 19:02:32.320345 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.318367    2561 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 19:02:32.320345 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.318381    2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 19:02:32.320345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318452    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:32.320345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318459    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:32.320345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318462    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:32.320345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318466    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:32.320345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318469    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:32.320345 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318472    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318475    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318479    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318495    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318498    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318502    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318504    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318508    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318514    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318518    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318522    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318525    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318527    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318530    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318533    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318536    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318538    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318541    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318544    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318547    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:32.320750 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318549    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318552    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318556    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318559    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318564    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318567    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318570    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318573    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318575    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318578    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318581    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318583    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318586    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318588    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318593    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318596    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318599    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318602    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318605    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:32.321236 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318608    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318611    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318614    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318617    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318620    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318623    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318625    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318628    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318630    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318633    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318635    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318638    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318640    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318643    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318646    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318648    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318651    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318654    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318658    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318661    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:32.321730 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318664    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318667    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318669    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318672    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318675    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318677    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318680    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318682    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318685    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318687    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318690    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318693    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318695    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318698    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318701    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318703    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318706    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318709    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318712    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:32.322257 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318714    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:32.322855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318717    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:32.322855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318720    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:32.322855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318722    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:32.322855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318724    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:32.322855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318727    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:32.322855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318730    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:32.322855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318732    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:32.322855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318735    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:32.322855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318738    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:32.322855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318741    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:32.322855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318745    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:32.322855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318749    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:32.322855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318751    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:32.322855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318754    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:32.322855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318757    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:32.323240 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.318763    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:02:32.323240 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318895    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:02:32.323240 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318900    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:02:32.323240 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318904    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:02:32.323240 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318907    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:02:32.323240 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318910    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:02:32.323240 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318913    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:02:32.323240 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318916    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:02:32.323240 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318919    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:02:32.323240 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318922    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:02:32.323240 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318926    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:02:32.323240 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318928    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:02:32.323240 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318932    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:02:32.323240 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318935    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318937    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318940    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318943    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318946    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318948    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318951    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318953    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318956    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318959    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318961    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318964    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318967    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318969    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318972    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318975    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318978    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318981    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318984    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318987    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:02:32.323603 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318990    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318992    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318995    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.318997    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319000    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319003    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319006    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319008    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319011    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319013    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319016    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319019    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319021    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319024    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319027    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319030    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319032    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319035    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319037    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319040    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:02:32.324087 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319043    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319045    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319048    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319050    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319053    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319055    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319059    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319063    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319066    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319069    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319072    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319075    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319077    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319080    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319082    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319085    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319087    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319090    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319093    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319095    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:02:32.324604 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319098    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319100    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319102    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319106    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319110    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319113    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319116    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319119    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319122    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319125    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319128    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319130    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319134    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319138    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319141    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319143    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319146    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319149    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319152    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:02:32.325128 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319154    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:02:32.325623 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319157    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:02:32.325623 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319160    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:02:32.325623 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319163    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:02:32.325623 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319166    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:02:32.325623 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319168    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:02:32.325623 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319171    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:02:32.325623 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:32.319173    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:02:32.325623 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.319179    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:02:32.325623 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.319339    2561 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 19:02:32.325623 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.321600    2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 19:02:32.325623 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.322825    2561 server.go:1019] "Starting client certificate rotation"
Jul 28 19:02:32.325623 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.322933    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 19:02:32.325623 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.322967    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 19:02:32.325953 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.324956    2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 19:02:32.326199 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.326177    2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 19:02:32.334170 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.334147    2561 log.go:25] "Validated CRI v1 runtime API"
Jul 28 19:02:32.336273 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.336253    2561 log.go:25] "Validated CRI v1 image API"
Jul 28 19:02:32.340040 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.340012    2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 19:02:32.342374 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.342350    2561 fs.go:135] Filesystem UUIDs: map[405e2bba-3aaf-4622-8d42-c04c583efd85:/dev/nvme0n1p4 605fb0c1-3d90-40d6-b5fc-7c84216f4c56:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 28 19:02:32.342443 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.342374    2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 19:02:32.349290 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.349152    2561 manager.go:216] Machine: {Timestamp:2026-07-28 19:02:32.346513893 +0000 UTC m=+0.292998771 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097089 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2027e8fad1045b5253750e96899aaf SystemUUID:ec2027e8-fad1-045b-5253-750e96899aaf BootID:a84a526f-c02d-43d4-9538-82b40600ec70 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6d:e2:04:5f:7b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6d:e2:04:5f:7b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:89:11:2d:f8:eb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 19:02:32.349290 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.349288    2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 19:02:32.349451 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.349403    2561 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 19:02:32.350772 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.350753    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 19:02:32.352127 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.352098    2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 19:02:32.352275 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.352131    2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-70.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 19:02:32.352741 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.352728    2561 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 19:02:32.352786 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.352747    2561 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 19:02:32.352786 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.352757    2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 19:02:32.353332 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.353319    2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 19:02:32.354020 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.354005    2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 19:02:32.355664 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.355649    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 19:02:32.355813 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.355803    2561 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 19:02:32.355853 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.355835    2561 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 19:02:32.355853 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.355848    2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 19:02:32.355906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.355860    2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 19:02:32.355906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.355873    2561 kubelet.go:398] "Adding apiserver pod source"
Jul 28 19:02:32.355906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.355892    2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 19:02:32.356377 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.356363    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 19:02:32.356449 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.356385    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 19:02:32.358307 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.358287    2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 19:02:32.359902 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.359887    2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 19:02:32.359964 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.359919    2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 19:02:32.361280 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.361270    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 19:02:32.361342 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.361285    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 19:02:32.361342 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.361291    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 19:02:32.361342 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.361297    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 19:02:32.361342 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.361306    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 19:02:32.361342 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.361312    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 19:02:32.361342 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.361323    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 19:02:32.361342 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.361330    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 19:02:32.361342 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.361337    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 19:02:32.361342 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.361342    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 19:02:32.361598 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.361361    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 19:02:32.361598 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.361367    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 19:02:32.362071 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.362050    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 19:02:32.362109 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.362073    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 19:02:32.365544 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.365530    2561 server.go:1268] "Started kubelet"
Jul 28 19:02:32.365693 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.365624    2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 19:02:32.365754 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.365675    2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 19:02:32.365754 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.365721    2561 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 19:02:32.366610 ip-10-0-137-70 systemd[1]: Started Kubernetes Kubelet.
Jul 28 19:02:32.370286 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.370259    2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 19:02:32.370443 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.370365    2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 19:02:32.370638 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.370621    2561 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 19:02:32.372414 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.372389    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-70.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 19:02:32.372535 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.372434    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 19:02:32.373118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.373102    2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-70.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 19:02:32.373163 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.372368    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-70.ec2.internal.18c68961158e9132  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-70.ec2.internal,UID:ip-10-0-137-70.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-70.ec2.internal,},FirstTimestamp:2026-07-28 19:02:32.365478194 +0000 UTC m=+0.311963053,LastTimestamp:2026-07-28 19:02:32.365478194 +0000 UTC m=+0.311963053,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-70.ec2.internal,}"
Jul 28 19:02:32.375454 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.375435    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lp8lz"
Jul 28 19:02:32.375892 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.375869    2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 19:02:32.375892 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.375884    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 19:02:32.376018 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.375951    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:32.376466 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.376433    2561 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 19:02:32.376466 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.376451    2561 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 19:02:32.376628 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.376434    2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 19:02:32.376628 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.376625    2561 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 19:02:32.376722 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.376632    2561 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 19:02:32.378287 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.378173    2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-70.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 19:02:32.378287 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.378264    2561 factory.go:55] Registering systemd factory
Jul 28 19:02:32.378287 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.378279    2561 factory.go:223] Registration of the systemd container factory successfully
Jul 28 19:02:32.378442 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.378270    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 19:02:32.378518 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.378503    2561 factory.go:153] Registering CRI-O factory
Jul 28 19:02:32.378518 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.378516    2561 factory.go:223] Registration of the crio container factory successfully
Jul 28 19:02:32.378607 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.378577    2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 19:02:32.378607 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.378600    2561 factory.go:103] Registering Raw factory
Jul 28 19:02:32.378699 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.378614    2561 manager.go:1196] Started watching for new ooms in manager
Jul 28 19:02:32.379284 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.379189    2561 manager.go:349] Starting recovery of all containers
Jul 28 19:02:32.381676 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.381650    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lp8lz"
Jul 28 19:02:32.389408 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.389386    2561 manager.go:354] Recovery completed
Jul 28 19:02:32.394166 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.394150    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:32.400179 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.400006    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:32.400311 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.400191    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:32.400311 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.400203    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:32.400755 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.400741    2561 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 19:02:32.400755 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.400754    2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 19:02:32.400843 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.400770    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 19:02:32.400875 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.400844    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:32.402348 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.402336    2561 policy_none.go:49] "None policy: Start"
Jul 28 19:02:32.402382 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.402353    2561 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 19:02:32.402382 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.402363    2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 19:02:32.402804 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.402701    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-70.ec2.internal.18c6896117a0058e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-70.ec2.internal,UID:ip-10-0-137-70.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-70.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-70.ec2.internal,},FirstTimestamp:2026-07-28 19:02:32.400176526 +0000 UTC m=+0.346661388,LastTimestamp:2026-07-28 19:02:32.400176526 +0000 UTC m=+0.346661388,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-70.ec2.internal,}"
Jul 28 19:02:32.402981 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.402966    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:32.403022 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.402993    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:32.403022 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.403009    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:32.403246 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.403234    2561 policy_none.go:47] "Start"
Jul 28 19:02:32.442480 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.442460    2561 manager.go:337] "Starting Device Plugin manager"
Jul 28 19:02:32.442658 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.442521    2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 19:02:32.442658 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.442535    2561 server.go:87] "Starting device plugin registration server"
Jul 28 19:02:32.442881 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.442868    2561 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 19:02:32.442932 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.442885    2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 19:02:32.443038 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.443015    2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 19:02:32.443154 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.443106    2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 19:02:32.443154 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.443114    2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 19:02:32.443720 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.443696    2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 19:02:32.443819 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.443761    2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:32.543205 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.543165    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:32.544334 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.544264    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:32.544334 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.544299    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:32.544334 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.544313    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:32.544484 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.544350    2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-70.ec2.internal"
Jul 28 19:02:32.553247 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.553188    2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-70.ec2.internal"
Jul 28 19:02:32.553247 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.553221    2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-70.ec2.internal\": node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:32.572360 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.572325    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:32.595923 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.595879    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 19:02:32.597419 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.597392    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 19:02:32.597554 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.597436    2561 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 19:02:32.597554 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.597467    2561 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 19:02:32.597554 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.597507    2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 19:02:32.600233 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.600206    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:02:32.673033 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.672994    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:32.698470 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.698428    2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-70.ec2.internal"]
Jul 28 19:02:32.698580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.698556    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:32.699697 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.699678    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:32.699804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.699710    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:32.699804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.699726    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:32.700915 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.700901    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:32.701091 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.701073    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal"
Jul 28 19:02:32.701140 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.701112    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:32.701793 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.701773    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:32.701893 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.701805    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:32.701893 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.701815    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:32.702242 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.702222    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:32.702341 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.702248    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:32.702341 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.702259    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:32.703503 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.703489    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-70.ec2.internal"
Jul 28 19:02:32.703558 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.703518    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:02:32.705761 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.705743    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:02:32.705866 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.705774    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:02:32.705866 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.705786    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:02:32.721372 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.721334    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-70.ec2.internal\" not found" node="ip-10-0-137-70.ec2.internal"
Jul 28 19:02:32.726209 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.726187    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-70.ec2.internal\" not found" node="ip-10-0-137-70.ec2.internal"
Jul 28 19:02:32.773737 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.773695    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:32.780389 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.780355    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eb8f3a29ea7328f96977f4eca0275487-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal\" (UID: \"eb8f3a29ea7328f96977f4eca0275487\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal"
Jul 28 19:02:32.780544 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.780395    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb8f3a29ea7328f96977f4eca0275487-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal\" (UID: \"eb8f3a29ea7328f96977f4eca0275487\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal"
Jul 28 19:02:32.780544 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.780416    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/683f92978e5f7131a5a5e68a817885c9-config\") pod \"kube-apiserver-proxy-ip-10-0-137-70.ec2.internal\" (UID: \"683f92978e5f7131a5a5e68a817885c9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-70.ec2.internal"
Jul 28 19:02:32.874183 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.874081    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:32.881073 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.881038    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eb8f3a29ea7328f96977f4eca0275487-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal\" (UID: \"eb8f3a29ea7328f96977f4eca0275487\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal"
Jul 28 19:02:32.881144 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.881076    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb8f3a29ea7328f96977f4eca0275487-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal\" (UID: \"eb8f3a29ea7328f96977f4eca0275487\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal"
Jul 28 19:02:32.881144 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.881095    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/683f92978e5f7131a5a5e68a817885c9-config\") pod \"kube-apiserver-proxy-ip-10-0-137-70.ec2.internal\" (UID: \"683f92978e5f7131a5a5e68a817885c9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-70.ec2.internal"
Jul 28 19:02:32.881218 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.881163    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/683f92978e5f7131a5a5e68a817885c9-config\") pod \"kube-apiserver-proxy-ip-10-0-137-70.ec2.internal\" (UID: \"683f92978e5f7131a5a5e68a817885c9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-70.ec2.internal"
Jul 28 19:02:32.881218 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.881173    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eb8f3a29ea7328f96977f4eca0275487-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal\" (UID: \"eb8f3a29ea7328f96977f4eca0275487\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal"
Jul 28 19:02:32.881218 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:32.881165    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb8f3a29ea7328f96977f4eca0275487-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal\" (UID: \"eb8f3a29ea7328f96977f4eca0275487\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal"
Jul 28 19:02:32.981938 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:32.981871    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:33.026480 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:33.026429    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal"
Jul 28 19:02:33.029174 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:33.029145    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-70.ec2.internal"
Jul 28 19:02:33.082318 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:33.082285    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:33.182482 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:33.182395    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:33.283586 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:33.283541    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:33.324895 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:33.324851    2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 19:02:33.325476 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:33.325081    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 19:02:33.383728 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:33.383695    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:33.385883 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:33.385859    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 18:57:32 +0000 UTC" deadline="2028-01-16 04:03:46.549374412 +0000 UTC"
Jul 28 19:02:33.385883 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:33.385883    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12873h1m13.163494157s"
Jul 28 19:02:33.484864 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:33.484757    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:33.509108 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:33.509079    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:02:33.585693 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:33.585656    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:33.686541 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:33.686498    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:33.739245 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:33.739171    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb8f3a29ea7328f96977f4eca0275487.slice/crio-f6a6d20b4a34445389cfe630b06967d5a54a7578c27c86e9b8693ce539ea3c99 WatchSource:0}: Error finding container f6a6d20b4a34445389cfe630b06967d5a54a7578c27c86e9b8693ce539ea3c99: Status 404 returned error can't find the container with id f6a6d20b4a34445389cfe630b06967d5a54a7578c27c86e9b8693ce539ea3c99
Jul 28 19:02:33.744011 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:33.743990    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:02:33.744429 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:33.744403    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod683f92978e5f7131a5a5e68a817885c9.slice/crio-368e4fb51f455fac799befd1567b21501e34c8a3def946d2d66f41f0ba754211 WatchSource:0}: Error finding container 368e4fb51f455fac799befd1567b21501e34c8a3def946d2d66f41f0ba754211: Status 404 returned error can't find the container with id 368e4fb51f455fac799befd1567b21501e34c8a3def946d2d66f41f0ba754211
Jul 28 19:02:33.787128 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:33.787069    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:33.865075 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:33.865046    2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:02:33.887657 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:33.887611    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-70.ec2.internal\" not found"
Jul 28 19:02:33.898795 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:33.897315    2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:02:33.980183 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:33.980115    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-70.ec2.internal"
Jul 28 19:02:33.991502 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:33.991413    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 19:02:33.992569 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:33.992546    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal"
Jul 28 19:02:34.002943 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.002902    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 19:02:34.359215 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.359116    2561 apiserver.go:52] "Watching apiserver"
Jul 28 19:02:34.365446 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.365417    2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 19:02:34.366442 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.366415    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-25rm8","openshift-multus/multus-rbq99","kube-system/kube-apiserver-proxy-ip-10-0-137-70.ec2.internal","openshift-cluster-node-tuning-operator/tuned-tks98","openshift-multus/network-metrics-daemon-hntnt","openshift-network-diagnostics/network-check-target-wq7g6","openshift-network-operator/iptables-alerter-nbdhf","openshift-ovn-kubernetes/ovnkube-node-wcdpw","kube-system/global-pull-secret-syncer-68mlb","kube-system/konnectivity-agent-lxcmx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t","openshift-dns/node-resolver-j5pqn","openshift-image-registry/node-ca-ffk9d","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal"]
Jul 28 19:02:34.370472 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.370440    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nbdhf"
Jul 28 19:02:34.372803 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.372780    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ffk9d"
Jul 28 19:02:34.373310 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.373287    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fkv95\""
Jul 28 19:02:34.373416 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.373316    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:02:34.373416 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.373292    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 19:02:34.373416 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.373347    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 19:02:34.375016 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.374991    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 19:02:34.375126 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.375027    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 19:02:34.375193 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.375130    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.375499 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.375294    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 19:02:34.375624 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.375598    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nrqz4\""
Jul 28 19:02:34.376775 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.376750    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 19:02:34.378199 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.377697    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.378267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.378211    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q2rxx\""
Jul 28 19:02:34.378519 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.378352    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 19:02:34.378519 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.378420    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 19:02:34.378519 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.378453    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 19:02:34.378519 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.378466    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 19:02:34.378813 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.378776    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 19:02:34.378892 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.378874    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 19:02:34.379966 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.379950    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:02:34.380338 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.380321    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cpzxr\""
Jul 28 19:02:34.380405 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.380325    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:34.380479 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:34.380451    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:02:34.380577 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.380557    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 19:02:34.382810 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.382788    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:34.382933 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:34.382886    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:02:34.385535 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.385507    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:34.385620 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:34.385585    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:02:34.390380 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.390321    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.391547 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.390824    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e45507b7-c87c-44b9-b495-b5d812d57324-host\") pod \"node-ca-ffk9d\" (UID: \"e45507b7-c87c-44b9-b495-b5d812d57324\") " pod="openshift-image-registry/node-ca-ffk9d"
Jul 28 19:02:34.391547 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.390924    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e45507b7-c87c-44b9-b495-b5d812d57324-serviceca\") pod \"node-ca-ffk9d\" (UID: \"e45507b7-c87c-44b9-b495-b5d812d57324\") " pod="openshift-image-registry/node-ca-ffk9d"
Jul 28 19:02:34.391547 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.390992    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-var-lib-openvswitch\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.391547 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391061    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-run-openvswitch\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.391547 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391126    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-run-ovn-kubernetes\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.391547 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391162    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-modprobe-d\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.391547 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391198    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-run-netns\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.391547 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391233    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-kubelet\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.391547 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391265    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-systemd-units\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.391547 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391339    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-etc-openvswitch\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.391547 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391393    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-run-ovn\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.391547 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391437    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.391547 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391503    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-host\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.391547 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391534    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee2de49f-f6cf-4709-b4b9-f75215cbc196-host-slash\") pod \"iptables-alerter-nbdhf\" (UID: \"ee2de49f-f6cf-4709-b4b9-f75215cbc196\") " pod="openshift-network-operator/iptables-alerter-nbdhf"
Jul 28 19:02:34.392316 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391617    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-slash\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.392316 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391667    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-node-log\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.392316 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391705    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bde3f901-3495-4178-bedc-62e92c8aaecc-ovnkube-config\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.392316 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391765    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-kubernetes\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.392316 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391845    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-systemd\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.392316 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391955    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-lib-modules\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.392316 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.391998    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fb717f7-11df-4903-9208-4e7d35398eb1-tmp\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.392316 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392075    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-run-systemd\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.392316 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392139    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f94vp\" (UniqueName: \"kubernetes.io/projected/ee2de49f-f6cf-4709-b4b9-f75215cbc196-kube-api-access-f94vp\") pod \"iptables-alerter-nbdhf\" (UID: \"ee2de49f-f6cf-4709-b4b9-f75215cbc196\") " pod="openshift-network-operator/iptables-alerter-nbdhf"
Jul 28 19:02:34.392316 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392186    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-log-socket\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.392316 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392221    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bde3f901-3495-4178-bedc-62e92c8aaecc-ovn-node-metrics-cert\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.392316 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392240    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz48z\" (UniqueName: \"kubernetes.io/projected/bde3f901-3495-4178-bedc-62e92c8aaecc-kube-api-access-pz48z\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.392316 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392265    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-var-lib-kubelet\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.392951 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392330    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-cni-netd\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.392951 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392349    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-sysconfig\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.392951 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392777    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-tuned\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.392951 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392819    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bde3f901-3495-4178-bedc-62e92c8aaecc-ovnkube-script-lib\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.392951 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392847    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzzw6\" (UniqueName: \"kubernetes.io/projected/4fb717f7-11df-4903-9208-4e7d35398eb1-kube-api-access-rzzw6\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.392951 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392873    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-cni-bin\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.392951 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392896    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bde3f901-3495-4178-bedc-62e92c8aaecc-env-overrides\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.392951 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392920    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-run\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.392951 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392947    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee2de49f-f6cf-4709-b4b9-f75215cbc196-iptables-alerter-script\") pod \"iptables-alerter-nbdhf\" (UID: \"ee2de49f-f6cf-4709-b4b9-f75215cbc196\") " pod="openshift-network-operator/iptables-alerter-nbdhf"
Jul 28 19:02:34.393437 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392961    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 19:02:34.393437 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392973    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tvqx\" (UniqueName: \"kubernetes.io/projected/e45507b7-c87c-44b9-b495-b5d812d57324-kube-api-access-4tvqx\") pod \"node-ca-ffk9d\" (UID: \"e45507b7-c87c-44b9-b495-b5d812d57324\") " pod="openshift-image-registry/node-ca-ffk9d"
Jul 28 19:02:34.393437 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.392997    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-sysctl-d\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.393437 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.393022    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-sysctl-conf\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.393437 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.393050    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-sys\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.393437 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.393259    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.393437 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.393372    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 19:02:34.393823 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.393444    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-p7xcd\""
Jul 28 19:02:34.393911 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.393816    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 19:02:34.393911 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.393878    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 19:02:34.394042 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.393934    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 19:02:34.394042 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.393967    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 19:02:34.395698 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.395673    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 19:02:34.395861 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.395838    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lxcmx"
Jul 28 19:02:34.395935 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.395906    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vl5q9\""
Jul 28 19:02:34.398223 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.398203    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6sdsm\""
Jul 28 19:02:34.398342 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.398246    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 19:02:34.398342 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.398293    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 19:02:34.398550 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.398535    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.400892 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.400871    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 19:02:34.400991 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.400926    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j5pqn"
Jul 28 19:02:34.401125 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.401100    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 19:02:34.401241 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.401156    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jkfjf\""
Jul 28 19:02:34.401311 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.401227    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 19:02:34.403135 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.403116    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-glwzt\""
Jul 28 19:02:34.403359 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.403341    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 19:02:34.403452 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.403388    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 19:02:34.415115 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.415088    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fzlw7"
Jul 28 19:02:34.421739 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.421715    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fzlw7"
Jul 28 19:02:34.477535 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.477501    2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 19:02:34.493416 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.493368    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcmln\" (UniqueName: \"kubernetes.io/projected/1bc180f1-52ba-4d7f-a886-025e5a39af95-kube-api-access-tcmln\") pod \"node-resolver-j5pqn\" (UID: \"1bc180f1-52ba-4d7f-a886-025e5a39af95\") " pod="openshift-dns/node-resolver-j5pqn"
Jul 28 19:02:34.493644 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.493422    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bde3f901-3495-4178-bedc-62e92c8aaecc-ovnkube-script-lib\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.493644 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.493522    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-etc-selinux\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.493644 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.493564    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/563e6899-0366-4322-a41e-b59d7a4a4b12-dbus\") pod \"global-pull-secret-syncer-68mlb\" (UID: \"563e6899-0366-4322-a41e-b59d7a4a4b12\") " pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:34.493644 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.493619    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c12d37f-cd16-4f11-9681-eaab31e63548-cni-binary-copy\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.494543 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.494132    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee2de49f-f6cf-4709-b4b9-f75215cbc196-iptables-alerter-script\") pod \"iptables-alerter-nbdhf\" (UID: \"ee2de49f-f6cf-4709-b4b9-f75215cbc196\") " pod="openshift-network-operator/iptables-alerter-nbdhf"
Jul 28 19:02:34.494543 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.494210    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tvqx\" (UniqueName: \"kubernetes.io/projected/e45507b7-c87c-44b9-b495-b5d812d57324-kube-api-access-4tvqx\") pod \"node-ca-ffk9d\" (UID: \"e45507b7-c87c-44b9-b495-b5d812d57324\") " pod="openshift-image-registry/node-ca-ffk9d"
Jul 28 19:02:34.494543 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.494251    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-sysctl-conf\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.494543 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.494293    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-system-cni-dir\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.494543 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.494328    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-run-k8s-cni-cncf-io\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.494543 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.494364    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-hostroot\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.494910 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.494691    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-sysctl-conf\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.494910 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.494697    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bde3f901-3495-4178-bedc-62e92c8aaecc-ovnkube-script-lib\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.494910 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.494509    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3c12d37f-cd16-4f11-9681-eaab31e63548-multus-daemon-config\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.494910 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.494873    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e45507b7-c87c-44b9-b495-b5d812d57324-serviceca\") pod \"node-ca-ffk9d\" (UID: \"e45507b7-c87c-44b9-b495-b5d812d57324\") " pod="openshift-image-registry/node-ca-ffk9d"
Jul 28 19:02:34.495110 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.494939    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-var-lib-openvswitch\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.495110 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495004    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-run-ovn-kubernetes\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.495110 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495015    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee2de49f-f6cf-4709-b4b9-f75215cbc196-iptables-alerter-script\") pod \"iptables-alerter-nbdhf\" (UID: \"ee2de49f-f6cf-4709-b4b9-f75215cbc196\") " pod="openshift-network-operator/iptables-alerter-nbdhf"
Jul 28 19:02:34.495110 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495041    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/563e6899-0366-4322-a41e-b59d7a4a4b12-kubelet-config\") pod \"global-pull-secret-syncer-68mlb\" (UID: \"563e6899-0366-4322-a41e-b59d7a4a4b12\") " pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:34.495110 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495078    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-multus-conf-dir\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.495521 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495112    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-var-lib-openvswitch\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.495521 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495179    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/83181e12-1e53-4972-8b1b-6ab591b30cf7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.495521 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495260    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-systemd-units\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.495521 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495272    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-run-ovn-kubernetes\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.495521 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495299    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.495521 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495334    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-systemd-units\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.495521 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495331    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret\") pod \"global-pull-secret-syncer-68mlb\" (UID: \"563e6899-0366-4322-a41e-b59d7a4a4b12\") " pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:34.495521 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495373    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9cc9\" (UniqueName: \"kubernetes.io/projected/0b4c98d2-1b87-4f67-8881-b4e5639b9895-kube-api-access-c9cc9\") pod \"network-metrics-daemon-hntnt\" (UID: \"0b4c98d2-1b87-4f67-8881-b4e5639b9895\") " pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:34.495521 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495411    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-cnibin\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.495521 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495421    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.495521 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495442    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-var-lib-cni-bin\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.495521 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495516    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q77nh\" (UniqueName: \"kubernetes.io/projected/3c12d37f-cd16-4f11-9681-eaab31e63548-kube-api-access-q77nh\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.496145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495593    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee2de49f-f6cf-4709-b4b9-f75215cbc196-host-slash\") pod \"iptables-alerter-nbdhf\" (UID: \"ee2de49f-f6cf-4709-b4b9-f75215cbc196\") " pod="openshift-network-operator/iptables-alerter-nbdhf"
Jul 28 19:02:34.496145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495631    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-slash\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.496145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495705    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-node-log\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.496145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495782    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee2de49f-f6cf-4709-b4b9-f75215cbc196-host-slash\") pod \"iptables-alerter-nbdhf\" (UID: \"ee2de49f-f6cf-4709-b4b9-f75215cbc196\") " pod="openshift-network-operator/iptables-alerter-nbdhf"
Jul 28 19:02:34.496145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495801    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-slash\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.496145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495820    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bde3f901-3495-4178-bedc-62e92c8aaecc-ovnkube-config\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.496145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495856    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-kubernetes\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.496145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495892    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-systemd\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.496145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495944    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-lib-modules\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.496145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495947    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e45507b7-c87c-44b9-b495-b5d812d57324-serviceca\") pod \"node-ca-ffk9d\" (UID: \"e45507b7-c87c-44b9-b495-b5d812d57324\") " pod="openshift-image-registry/node-ca-ffk9d"
Jul 28 19:02:34.496145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.495980    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-run-netns\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.496145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496016    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-node-log\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.496145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496018    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-run-systemd\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.496145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496070    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-log-socket\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.496145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496077    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-run-systemd\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496142    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-systemd\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496158    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-kubernetes\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496242    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-log-socket\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496284    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-etc-kubernetes\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496324    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-lib-modules\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496324    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-cni-netd\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496362    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bde3f901-3495-4178-bedc-62e92c8aaecc-ovnkube-config\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496371    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/83181e12-1e53-4972-8b1b-6ab591b30cf7-system-cni-dir\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496427    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twhwc\" (UniqueName: \"kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc\") pod \"network-check-target-wq7g6\" (UID: \"ed17b640-5b2f-4ba7-ac59-1476dfa7ce22\") " pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496459    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-cni-netd\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496456    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-device-dir\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496512    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzzw6\" (UniqueName: \"kubernetes.io/projected/4fb717f7-11df-4903-9208-4e7d35398eb1-kube-api-access-rzzw6\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496547    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8mld\" (UniqueName: \"kubernetes.io/projected/83181e12-1e53-4972-8b1b-6ab591b30cf7-kube-api-access-f8mld\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496591    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1bc180f1-52ba-4d7f-a886-025e5a39af95-hosts-file\") pod \"node-resolver-j5pqn\" (UID: \"1bc180f1-52ba-4d7f-a886-025e5a39af95\") " pod="openshift-dns/node-resolver-j5pqn"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496645    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-var-lib-kubelet\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496677    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnwgx\" (UniqueName: \"kubernetes.io/projected/c540efb2-a917-41b1-8b49-756ea74c1d68-kube-api-access-bnwgx\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.496873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496712    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-cni-bin\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496754    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bde3f901-3495-4178-bedc-62e92c8aaecc-env-overrides\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496778    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-run\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496896    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/40534a0a-48dc-48a6-bfd0-3d0eb410a4c1-konnectivity-ca\") pod \"konnectivity-agent-lxcmx\" (UID: \"40534a0a-48dc-48a6-bfd0-3d0eb410a4c1\") " pod="kube-system/konnectivity-agent-lxcmx"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496932    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-sysctl-d\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.496977    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-sys\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497005    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/83181e12-1e53-4972-8b1b-6ab591b30cf7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497037    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/83181e12-1e53-4972-8b1b-6ab591b30cf7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497079    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bc180f1-52ba-4d7f-a886-025e5a39af95-tmp-dir\") pod \"node-resolver-j5pqn\" (UID: \"1bc180f1-52ba-4d7f-a886-025e5a39af95\") " pod="openshift-dns/node-resolver-j5pqn"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497114    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-sys-fs\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497148    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e45507b7-c87c-44b9-b495-b5d812d57324-host\") pod \"node-ca-ffk9d\" (UID: \"e45507b7-c87c-44b9-b495-b5d812d57324\") " pod="openshift-image-registry/node-ca-ffk9d"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497179    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-run-openvswitch\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497290    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e45507b7-c87c-44b9-b495-b5d812d57324-host\") pod \"node-ca-ffk9d\" (UID: \"e45507b7-c87c-44b9-b495-b5d812d57324\") " pod="openshift-image-registry/node-ca-ffk9d"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497328    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-modprobe-d\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497375    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/83181e12-1e53-4972-8b1b-6ab591b30cf7-os-release\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497424    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-multus-socket-dir-parent\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497455    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/83181e12-1e53-4972-8b1b-6ab591b30cf7-cnibin\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.497675 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497487    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/83181e12-1e53-4972-8b1b-6ab591b30cf7-cni-binary-copy\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497522    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-run-netns\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497550    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-kubelet\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497599    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-etc-openvswitch\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497631    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-run-ovn\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497661    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-host\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497691    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-var-lib-cni-multus\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497719    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fb717f7-11df-4903-9208-4e7d35398eb1-tmp\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497752    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/40534a0a-48dc-48a6-bfd0-3d0eb410a4c1-agent-certs\") pod \"konnectivity-agent-lxcmx\" (UID: \"40534a0a-48dc-48a6-bfd0-3d0eb410a4c1\") " pod="kube-system/konnectivity-agent-lxcmx"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497783    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs\") pod \"network-metrics-daemon-hntnt\" (UID: \"0b4c98d2-1b87-4f67-8881-b4e5639b9895\") " pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497820    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-multus-cni-dir\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497848    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-os-release\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497881    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-run-multus-certs\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.497920    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-socket-dir\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498111    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-registration-dir\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498151    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f94vp\" (UniqueName: \"kubernetes.io/projected/ee2de49f-f6cf-4709-b4b9-f75215cbc196-kube-api-access-f94vp\") pod \"iptables-alerter-nbdhf\" (UID: \"ee2de49f-f6cf-4709-b4b9-f75215cbc196\") " pod="openshift-network-operator/iptables-alerter-nbdhf"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498214    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-run-openvswitch\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.498406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498255    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bde3f901-3495-4178-bedc-62e92c8aaecc-ovn-node-metrics-cert\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498241    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-run\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498358    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-run-ovn\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498473    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-modprobe-d\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498615    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-cni-bin\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498655    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-sysctl-d\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498710    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-run-netns\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498717    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-host-kubelet\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498752    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bde3f901-3495-4178-bedc-62e92c8aaecc-env-overrides\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498769    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bde3f901-3495-4178-bedc-62e92c8aaecc-etc-openvswitch\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498830    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-sys\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498891    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-host\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498963    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz48z\" (UniqueName: \"kubernetes.io/projected/bde3f901-3495-4178-bedc-62e92c8aaecc-kube-api-access-pz48z\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.498997    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-var-lib-kubelet\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.499036    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.499129    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-sysconfig\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.499177    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-tuned\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.499267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.499198    2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 19:02:34.499897 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.499402    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-sysconfig\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.499897 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.499435    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fb717f7-11df-4903-9208-4e7d35398eb1-var-lib-kubelet\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.503033 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.502999    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4fb717f7-11df-4903-9208-4e7d35398eb1-etc-tuned\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.503133 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.503076    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fb717f7-11df-4903-9208-4e7d35398eb1-tmp\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.503198 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.503178    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bde3f901-3495-4178-bedc-62e92c8aaecc-ovn-node-metrics-cert\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.503574 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.503527    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tvqx\" (UniqueName: \"kubernetes.io/projected/e45507b7-c87c-44b9-b495-b5d812d57324-kube-api-access-4tvqx\") pod \"node-ca-ffk9d\" (UID: \"e45507b7-c87c-44b9-b495-b5d812d57324\") " pod="openshift-image-registry/node-ca-ffk9d"
Jul 28 19:02:34.507673 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.507642    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzzw6\" (UniqueName: \"kubernetes.io/projected/4fb717f7-11df-4903-9208-4e7d35398eb1-kube-api-access-rzzw6\") pod \"tuned-tks98\" (UID: \"4fb717f7-11df-4903-9208-4e7d35398eb1\") " pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.507803 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.507680    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz48z\" (UniqueName: \"kubernetes.io/projected/bde3f901-3495-4178-bedc-62e92c8aaecc-kube-api-access-pz48z\") pod \"ovnkube-node-wcdpw\" (UID: \"bde3f901-3495-4178-bedc-62e92c8aaecc\") " pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.507803 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.507704    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f94vp\" (UniqueName: \"kubernetes.io/projected/ee2de49f-f6cf-4709-b4b9-f75215cbc196-kube-api-access-f94vp\") pod \"iptables-alerter-nbdhf\" (UID: \"ee2de49f-f6cf-4709-b4b9-f75215cbc196\") " pod="openshift-network-operator/iptables-alerter-nbdhf"
Jul 28 19:02:34.600109 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600059    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-system-cni-dir\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.600266 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600117    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-run-k8s-cni-cncf-io\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.600266 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600143    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-hostroot\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.600266 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600191    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-system-cni-dir\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.600266 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600234    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-hostroot\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.600266 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600189    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3c12d37f-cd16-4f11-9681-eaab31e63548-multus-daemon-config\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.600533 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600191    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-run-k8s-cni-cncf-io\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.600533 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600316    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/563e6899-0366-4322-a41e-b59d7a4a4b12-kubelet-config\") pod \"global-pull-secret-syncer-68mlb\" (UID: \"563e6899-0366-4322-a41e-b59d7a4a4b12\") " pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:34.600533 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600345    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-multus-conf-dir\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.600533 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600373    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/83181e12-1e53-4972-8b1b-6ab591b30cf7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.600533 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600476    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret\") pod \"global-pull-secret-syncer-68mlb\" (UID: \"563e6899-0366-4322-a41e-b59d7a4a4b12\") " pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:34.600533 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600411    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-multus-conf-dir\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.600764 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600523    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9cc9\" (UniqueName: \"kubernetes.io/projected/0b4c98d2-1b87-4f67-8881-b4e5639b9895-kube-api-access-c9cc9\") pod \"network-metrics-daemon-hntnt\" (UID: \"0b4c98d2-1b87-4f67-8881-b4e5639b9895\") " pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:34.600764 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600590    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-cnibin\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.600764 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600613    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-var-lib-cni-bin\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.600764 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600638    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q77nh\" (UniqueName: \"kubernetes.io/projected/3c12d37f-cd16-4f11-9681-eaab31e63548-kube-api-access-q77nh\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.600764 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600661    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-run-netns\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.600764 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600678    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-etc-kubernetes\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.600764 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600695    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/83181e12-1e53-4972-8b1b-6ab591b30cf7-system-cni-dir\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.600764 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600720    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twhwc\" (UniqueName: \"kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc\") pod \"network-check-target-wq7g6\" (UID: \"ed17b640-5b2f-4ba7-ac59-1476dfa7ce22\") " pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:34.600764 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600737    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-device-dir\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600777    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8mld\" (UniqueName: \"kubernetes.io/projected/83181e12-1e53-4972-8b1b-6ab591b30cf7-kube-api-access-f8mld\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600804    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1bc180f1-52ba-4d7f-a886-025e5a39af95-hosts-file\") pod \"node-resolver-j5pqn\" (UID: \"1bc180f1-52ba-4d7f-a886-025e5a39af95\") " pod="openshift-dns/node-resolver-j5pqn"
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600824    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-var-lib-kubelet\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600841    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnwgx\" (UniqueName: \"kubernetes.io/projected/c540efb2-a917-41b1-8b49-756ea74c1d68-kube-api-access-bnwgx\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600859    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/40534a0a-48dc-48a6-bfd0-3d0eb410a4c1-konnectivity-ca\") pod \"konnectivity-agent-lxcmx\" (UID: \"40534a0a-48dc-48a6-bfd0-3d0eb410a4c1\") " pod="kube-system/konnectivity-agent-lxcmx"
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:34.600864    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600868    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3c12d37f-cd16-4f11-9681-eaab31e63548-multus-daemon-config\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600879    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/83181e12-1e53-4972-8b1b-6ab591b30cf7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600903    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/83181e12-1e53-4972-8b1b-6ab591b30cf7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:34.600939    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret podName:563e6899-0366-4322-a41e-b59d7a4a4b12 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:35.100916009 +0000 UTC m=+3.047400876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret") pod "global-pull-secret-syncer-68mlb" (UID: "563e6899-0366-4322-a41e-b59d7a4a4b12") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600972    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-cnibin\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600965    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bc180f1-52ba-4d7f-a886-025e5a39af95-tmp-dir\") pod \"node-resolver-j5pqn\" (UID: \"1bc180f1-52ba-4d7f-a886-025e5a39af95\") " pod="openshift-dns/node-resolver-j5pqn"
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601022    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-var-lib-cni-bin\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601023    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-sys-fs\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601046    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/83181e12-1e53-4972-8b1b-6ab591b30cf7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601068    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/83181e12-1e53-4972-8b1b-6ab591b30cf7-os-release\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.601083 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601071    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-sys-fs\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601096    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-multus-socket-dir-parent\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601151    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-multus-socket-dir-parent\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.600434    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/563e6899-0366-4322-a41e-b59d7a4a4b12-kubelet-config\") pod \"global-pull-secret-syncer-68mlb\" (UID: \"563e6899-0366-4322-a41e-b59d7a4a4b12\") " pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601373    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bc180f1-52ba-4d7f-a886-025e5a39af95-tmp-dir\") pod \"node-resolver-j5pqn\" (UID: \"1bc180f1-52ba-4d7f-a886-025e5a39af95\") " pod="openshift-dns/node-resolver-j5pqn"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601411    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/83181e12-1e53-4972-8b1b-6ab591b30cf7-cnibin\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601440    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/83181e12-1e53-4972-8b1b-6ab591b30cf7-cni-binary-copy\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601443    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/83181e12-1e53-4972-8b1b-6ab591b30cf7-system-cni-dir\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601449    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/83181e12-1e53-4972-8b1b-6ab591b30cf7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601478    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-var-lib-cni-multus\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601493    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-run-netns\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601438    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-etc-kubernetes\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601511    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/40534a0a-48dc-48a6-bfd0-3d0eb410a4c1-agent-certs\") pod \"konnectivity-agent-lxcmx\" (UID: \"40534a0a-48dc-48a6-bfd0-3d0eb410a4c1\") " pod="kube-system/konnectivity-agent-lxcmx"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601540    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/83181e12-1e53-4972-8b1b-6ab591b30cf7-cnibin\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601540    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs\") pod \"network-metrics-daemon-hntnt\" (UID: \"0b4c98d2-1b87-4f67-8881-b4e5639b9895\") " pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601595    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/83181e12-1e53-4972-8b1b-6ab591b30cf7-os-release\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601621    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-multus-cni-dir\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:34.601634    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:34.601885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601650    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-os-release\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601669    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-var-lib-kubelet\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601685    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-run-multus-certs\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:34.601724    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs podName:0b4c98d2-1b87-4f67-8881-b4e5639b9895 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:35.101705222 +0000 UTC m=+3.048190072 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs") pod "network-metrics-daemon-hntnt" (UID: "0b4c98d2-1b87-4f67-8881-b4e5639b9895") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601786    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-var-lib-cni-multus\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601843    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-device-dir\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.601957    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/83181e12-1e53-4972-8b1b-6ab591b30cf7-cni-binary-copy\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602141    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-multus-cni-dir\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602173    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-os-release\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602235    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3c12d37f-cd16-4f11-9681-eaab31e63548-host-run-multus-certs\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602316    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/83181e12-1e53-4972-8b1b-6ab591b30cf7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602318    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1bc180f1-52ba-4d7f-a886-025e5a39af95-hosts-file\") pod \"node-resolver-j5pqn\" (UID: \"1bc180f1-52ba-4d7f-a886-025e5a39af95\") " pod="openshift-dns/node-resolver-j5pqn"
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602340    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-socket-dir\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602379    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-registration-dir\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602413    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/40534a0a-48dc-48a6-bfd0-3d0eb410a4c1-konnectivity-ca\") pod \"konnectivity-agent-lxcmx\" (UID: \"40534a0a-48dc-48a6-bfd0-3d0eb410a4c1\") " pod="kube-system/konnectivity-agent-lxcmx"
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602439    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602488    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcmln\" (UniqueName: \"kubernetes.io/projected/1bc180f1-52ba-4d7f-a886-025e5a39af95-kube-api-access-tcmln\") pod \"node-resolver-j5pqn\" (UID: \"1bc180f1-52ba-4d7f-a886-025e5a39af95\") " pod="openshift-dns/node-resolver-j5pqn"
Jul 28 19:02:34.602781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602480    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-socket-dir\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.603522 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602534    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-etc-selinux\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.603522 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602558    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.603522 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602531    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-registration-dir\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.603522 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602587    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/563e6899-0366-4322-a41e-b59d7a4a4b12-dbus\") pod \"global-pull-secret-syncer-68mlb\" (UID: \"563e6899-0366-4322-a41e-b59d7a4a4b12\") " pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:34.603522 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602619    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c12d37f-cd16-4f11-9681-eaab31e63548-cni-binary-copy\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.603522 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602663    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c540efb2-a917-41b1-8b49-756ea74c1d68-etc-selinux\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.603522 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.602676    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/563e6899-0366-4322-a41e-b59d7a4a4b12-dbus\") pod \"global-pull-secret-syncer-68mlb\" (UID: \"563e6899-0366-4322-a41e-b59d7a4a4b12\") " pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:34.603821 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.603766    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c12d37f-cd16-4f11-9681-eaab31e63548-cni-binary-copy\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.603821 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.603763    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal" event={"ID":"eb8f3a29ea7328f96977f4eca0275487","Type":"ContainerStarted","Data":"f6a6d20b4a34445389cfe630b06967d5a54a7578c27c86e9b8693ce539ea3c99"}
Jul 28 19:02:34.604515 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.604488    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/40534a0a-48dc-48a6-bfd0-3d0eb410a4c1-agent-certs\") pod \"konnectivity-agent-lxcmx\" (UID: \"40534a0a-48dc-48a6-bfd0-3d0eb410a4c1\") " pod="kube-system/konnectivity-agent-lxcmx"
Jul 28 19:02:34.604971 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.604952    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-70.ec2.internal" event={"ID":"683f92978e5f7131a5a5e68a817885c9","Type":"ContainerStarted","Data":"368e4fb51f455fac799befd1567b21501e34c8a3def946d2d66f41f0ba754211"}
Jul 28 19:02:34.611828 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:34.611761    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:34.611828 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:34.611791    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:34.611828 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:34.611808    2561 projected.go:196] Error preparing data for projected volume kube-api-access-twhwc for pod openshift-network-diagnostics/network-check-target-wq7g6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:34.612096 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:34.611908    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc podName:ed17b640-5b2f-4ba7-ac59-1476dfa7ce22 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:35.111881877 +0000 UTC m=+3.058366750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-twhwc" (UniqueName: "kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc") pod "network-check-target-wq7g6" (UID: "ed17b640-5b2f-4ba7-ac59-1476dfa7ce22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:34.612601 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.612574    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9cc9\" (UniqueName: \"kubernetes.io/projected/0b4c98d2-1b87-4f67-8881-b4e5639b9895-kube-api-access-c9cc9\") pod \"network-metrics-daemon-hntnt\" (UID: \"0b4c98d2-1b87-4f67-8881-b4e5639b9895\") " pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:34.613421 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.613397    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8mld\" (UniqueName: \"kubernetes.io/projected/83181e12-1e53-4972-8b1b-6ab591b30cf7-kube-api-access-f8mld\") pod \"multus-additional-cni-plugins-25rm8\" (UID: \"83181e12-1e53-4972-8b1b-6ab591b30cf7\") " pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.613507 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.613456    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcmln\" (UniqueName: \"kubernetes.io/projected/1bc180f1-52ba-4d7f-a886-025e5a39af95-kube-api-access-tcmln\") pod \"node-resolver-j5pqn\" (UID: \"1bc180f1-52ba-4d7f-a886-025e5a39af95\") " pod="openshift-dns/node-resolver-j5pqn"
Jul 28 19:02:34.613639 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.613622    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnwgx\" (UniqueName: \"kubernetes.io/projected/c540efb2-a917-41b1-8b49-756ea74c1d68-kube-api-access-bnwgx\") pod \"aws-ebs-csi-driver-node-fk59t\" (UID: \"c540efb2-a917-41b1-8b49-756ea74c1d68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.613824 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.613807    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q77nh\" (UniqueName: \"kubernetes.io/projected/3c12d37f-cd16-4f11-9681-eaab31e63548-kube-api-access-q77nh\") pod \"multus-rbq99\" (UID: \"3c12d37f-cd16-4f11-9681-eaab31e63548\") " pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.682785 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.682741    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nbdhf"
Jul 28 19:02:34.689925 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.689896    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ffk9d"
Jul 28 19:02:34.694790 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:34.694756    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee2de49f_f6cf_4709_b4b9_f75215cbc196.slice/crio-98ae5933ed1ab01caaa74c943afd23deb3aead2b755821ff229aeed1b588d779 WatchSource:0}: Error finding container 98ae5933ed1ab01caaa74c943afd23deb3aead2b755821ff229aeed1b588d779: Status 404 returned error can't find the container with id 98ae5933ed1ab01caaa74c943afd23deb3aead2b755821ff229aeed1b588d779
Jul 28 19:02:34.700338 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.700308    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:02:34.701010 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:34.700985    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45507b7_c87c_44b9_b495_b5d812d57324.slice/crio-66677ce816887e732003e87ba546ed0ef118f89709b72fb353f193782b0f5f17 WatchSource:0}: Error finding container 66677ce816887e732003e87ba546ed0ef118f89709b72fb353f193782b0f5f17: Status 404 returned error can't find the container with id 66677ce816887e732003e87ba546ed0ef118f89709b72fb353f193782b0f5f17
Jul 28 19:02:34.705548 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.705524    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tks98"
Jul 28 19:02:34.711983 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.711953    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-25rm8"
Jul 28 19:02:34.714024 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:34.713963    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbde3f901_3495_4178_bedc_62e92c8aaecc.slice/crio-bcd6cf762626646d3964a09665d9f8264dbd858fdede15c544a6e1a73f5b8126 WatchSource:0}: Error finding container bcd6cf762626646d3964a09665d9f8264dbd858fdede15c544a6e1a73f5b8126: Status 404 returned error can't find the container with id bcd6cf762626646d3964a09665d9f8264dbd858fdede15c544a6e1a73f5b8126
Jul 28 19:02:34.716737 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:34.716707    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fb717f7_11df_4903_9208_4e7d35398eb1.slice/crio-d7929b1ca007eeb675313128d1b6a8c031de97fd7eac30d84b7209c407bbfc6b WatchSource:0}: Error finding container d7929b1ca007eeb675313128d1b6a8c031de97fd7eac30d84b7209c407bbfc6b: Status 404 returned error can't find the container with id d7929b1ca007eeb675313128d1b6a8c031de97fd7eac30d84b7209c407bbfc6b
Jul 28 19:02:34.717694 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.717677    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rbq99"
Jul 28 19:02:34.722618 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:34.722588    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83181e12_1e53_4972_8b1b_6ab591b30cf7.slice/crio-6a8caa0940861efec9987a1cca2a16e7b5f468f505c1f0208572e227a58ad0b0 WatchSource:0}: Error finding container 6a8caa0940861efec9987a1cca2a16e7b5f468f505c1f0208572e227a58ad0b0: Status 404 returned error can't find the container with id 6a8caa0940861efec9987a1cca2a16e7b5f468f505c1f0208572e227a58ad0b0
Jul 28 19:02:34.724651 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.724632    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lxcmx"
Jul 28 19:02:34.727895 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:34.727865    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c12d37f_cd16_4f11_9681_eaab31e63548.slice/crio-74cd0095fc649db1fe4e68e0b96d059659434c7664b91972fb39614b30e65964 WatchSource:0}: Error finding container 74cd0095fc649db1fe4e68e0b96d059659434c7664b91972fb39614b30e65964: Status 404 returned error can't find the container with id 74cd0095fc649db1fe4e68e0b96d059659434c7664b91972fb39614b30e65964
Jul 28 19:02:34.730543 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.730523    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t"
Jul 28 19:02:34.734862 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:34.734837    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40534a0a_48dc_48a6_bfd0_3d0eb410a4c1.slice/crio-b4273f6f807294ebede011cf8811954f9dbe03afddd0b0e7ecaef318c090c933 WatchSource:0}: Error finding container b4273f6f807294ebede011cf8811954f9dbe03afddd0b0e7ecaef318c090c933: Status 404 returned error can't find the container with id b4273f6f807294ebede011cf8811954f9dbe03afddd0b0e7ecaef318c090c933
Jul 28 19:02:34.735631 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.735612    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j5pqn"
Jul 28 19:02:34.740251 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:34.740222    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc540efb2_a917_41b1_8b49_756ea74c1d68.slice/crio-f08b59c6139baebb27372059e064671cdd42a8d3e18275858768c698a7f41d2e WatchSource:0}: Error finding container f08b59c6139baebb27372059e064671cdd42a8d3e18275858768c698a7f41d2e: Status 404 returned error can't find the container with id f08b59c6139baebb27372059e064671cdd42a8d3e18275858768c698a7f41d2e
Jul 28 19:02:34.745987 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:02:34.745957    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bc180f1_52ba_4d7f_a886_025e5a39af95.slice/crio-a735284dbc6a57764a7901bf40f9e2e7ced06c49f3e08342d41b7cbcd28b6ed5 WatchSource:0}: Error finding container a735284dbc6a57764a7901bf40f9e2e7ced06c49f3e08342d41b7cbcd28b6ed5: Status 404 returned error can't find the container with id a735284dbc6a57764a7901bf40f9e2e7ced06c49f3e08342d41b7cbcd28b6ed5
Jul 28 19:02:34.861100 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:34.861065    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:02:35.108785 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:35.108704    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs\") pod \"network-metrics-daemon-hntnt\" (UID: \"0b4c98d2-1b87-4f67-8881-b4e5639b9895\") " pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:35.108785 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:35.108752    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret\") pod \"global-pull-secret-syncer-68mlb\" (UID: \"563e6899-0366-4322-a41e-b59d7a4a4b12\") " pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:35.108986 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:35.108874    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:35.108986 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:35.108877    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:35.108986 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:35.108934    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret podName:563e6899-0366-4322-a41e-b59d7a4a4b12 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:36.108919398 +0000 UTC m=+4.055404246 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret") pod "global-pull-secret-syncer-68mlb" (UID: "563e6899-0366-4322-a41e-b59d7a4a4b12") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:35.108986 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:35.108948    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs podName:0b4c98d2-1b87-4f67-8881-b4e5639b9895 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:36.108941209 +0000 UTC m=+4.055426056 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs") pod "network-metrics-daemon-hntnt" (UID: "0b4c98d2-1b87-4f67-8881-b4e5639b9895") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:35.209683 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:35.209631    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twhwc\" (UniqueName: \"kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc\") pod \"network-check-target-wq7g6\" (UID: \"ed17b640-5b2f-4ba7-ac59-1476dfa7ce22\") " pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:35.209885 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:35.209818    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:35.209885 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:35.209846    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:35.209885 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:35.209861    2561 projected.go:196] Error preparing data for projected volume kube-api-access-twhwc for pod openshift-network-diagnostics/network-check-target-wq7g6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:35.215606 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:35.210345    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc podName:ed17b640-5b2f-4ba7-ac59-1476dfa7ce22 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:36.209917018 +0000 UTC m=+4.156401868 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-twhwc" (UniqueName: "kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc") pod "network-check-target-wq7g6" (UID: "ed17b640-5b2f-4ba7-ac59-1476dfa7ce22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:35.425525 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:35.425402    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 18:57:34 +0000 UTC" deadline="2028-04-28 14:07:03.059521 +0000 UTC"
Jul 28 19:02:35.425525 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:35.425470    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15355h4m27.634054932s"
Jul 28 19:02:35.598683 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:35.598635    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:35.598884 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:35.598782    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:02:35.599260 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:35.599238    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:35.599649 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:35.599352    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:02:35.633749 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:35.633705    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ffk9d" event={"ID":"e45507b7-c87c-44b9-b495-b5d812d57324","Type":"ContainerStarted","Data":"66677ce816887e732003e87ba546ed0ef118f89709b72fb353f193782b0f5f17"}
Jul 28 19:02:35.636223 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:35.636179    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nbdhf" event={"ID":"ee2de49f-f6cf-4709-b4b9-f75215cbc196","Type":"ContainerStarted","Data":"98ae5933ed1ab01caaa74c943afd23deb3aead2b755821ff229aeed1b588d779"}
Jul 28 19:02:35.640098 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:35.639991    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t" event={"ID":"c540efb2-a917-41b1-8b49-756ea74c1d68","Type":"ContainerStarted","Data":"f08b59c6139baebb27372059e064671cdd42a8d3e18275858768c698a7f41d2e"}
Jul 28 19:02:35.659429 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:35.659364    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25rm8" event={"ID":"83181e12-1e53-4972-8b1b-6ab591b30cf7","Type":"ContainerStarted","Data":"6a8caa0940861efec9987a1cca2a16e7b5f468f505c1f0208572e227a58ad0b0"}
Jul 28 19:02:35.667063 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:35.667017    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw" event={"ID":"bde3f901-3495-4178-bedc-62e92c8aaecc","Type":"ContainerStarted","Data":"bcd6cf762626646d3964a09665d9f8264dbd858fdede15c544a6e1a73f5b8126"}
Jul 28 19:02:35.671992 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:35.671901    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j5pqn" event={"ID":"1bc180f1-52ba-4d7f-a886-025e5a39af95","Type":"ContainerStarted","Data":"a735284dbc6a57764a7901bf40f9e2e7ced06c49f3e08342d41b7cbcd28b6ed5"}
Jul 28 19:02:35.686914 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:35.686810    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lxcmx" event={"ID":"40534a0a-48dc-48a6-bfd0-3d0eb410a4c1","Type":"ContainerStarted","Data":"b4273f6f807294ebede011cf8811954f9dbe03afddd0b0e7ecaef318c090c933"}
Jul 28 19:02:35.707203 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:35.707155    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rbq99" event={"ID":"3c12d37f-cd16-4f11-9681-eaab31e63548","Type":"ContainerStarted","Data":"74cd0095fc649db1fe4e68e0b96d059659434c7664b91972fb39614b30e65964"}
Jul 28 19:02:35.740924 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:35.740882    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tks98" event={"ID":"4fb717f7-11df-4903-9208-4e7d35398eb1","Type":"ContainerStarted","Data":"d7929b1ca007eeb675313128d1b6a8c031de97fd7eac30d84b7209c407bbfc6b"}
Jul 28 19:02:36.119620 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:36.119565    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs\") pod \"network-metrics-daemon-hntnt\" (UID: \"0b4c98d2-1b87-4f67-8881-b4e5639b9895\") " pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:36.119808 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:36.119656    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret\") pod \"global-pull-secret-syncer-68mlb\" (UID: \"563e6899-0366-4322-a41e-b59d7a4a4b12\") " pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:36.119887 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:36.119838    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:36.119942 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:36.119910    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret podName:563e6899-0366-4322-a41e-b59d7a4a4b12 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:38.119889324 +0000 UTC m=+6.066374187 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret") pod "global-pull-secret-syncer-68mlb" (UID: "563e6899-0366-4322-a41e-b59d7a4a4b12") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:36.124926 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:36.120328    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:36.124926 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:36.120383    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs podName:0b4c98d2-1b87-4f67-8881-b4e5639b9895 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:38.120366691 +0000 UTC m=+6.066851545 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs") pod "network-metrics-daemon-hntnt" (UID: "0b4c98d2-1b87-4f67-8881-b4e5639b9895") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:36.220912 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:36.220867    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twhwc\" (UniqueName: \"kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc\") pod \"network-check-target-wq7g6\" (UID: \"ed17b640-5b2f-4ba7-ac59-1476dfa7ce22\") " pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:36.221113 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:36.221081    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:36.221113 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:36.221114    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:36.221236 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:36.221128    2561 projected.go:196] Error preparing data for projected volume kube-api-access-twhwc for pod openshift-network-diagnostics/network-check-target-wq7g6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:36.221236 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:36.221191    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc podName:ed17b640-5b2f-4ba7-ac59-1476dfa7ce22 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:38.221171124 +0000 UTC m=+6.167655993 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-twhwc" (UniqueName: "kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc") pod "network-check-target-wq7g6" (UID: "ed17b640-5b2f-4ba7-ac59-1476dfa7ce22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:36.425862 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:36.425674    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 18:57:34 +0000 UTC" deadline="2028-02-07 09:57:06.803465776 +0000 UTC"
Jul 28 19:02:36.425862 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:36.425720    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13406h54m30.377750691s"
Jul 28 19:02:36.600473 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:36.600436    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:36.600698 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:36.600618    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:02:37.598773 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:37.598721    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:37.599337 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:37.598878    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:02:37.599337 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:37.599318    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:37.599537 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:37.599507    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:02:38.142021 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:38.141963    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret\") pod \"global-pull-secret-syncer-68mlb\" (UID: \"563e6899-0366-4322-a41e-b59d7a4a4b12\") " pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:38.142240 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:38.142068    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs\") pod \"network-metrics-daemon-hntnt\" (UID: \"0b4c98d2-1b87-4f67-8881-b4e5639b9895\") " pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:38.142240 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:38.142211    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:38.142360 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:38.142285    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs podName:0b4c98d2-1b87-4f67-8881-b4e5639b9895 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:42.14226238 +0000 UTC m=+10.088747228 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs") pod "network-metrics-daemon-hntnt" (UID: "0b4c98d2-1b87-4f67-8881-b4e5639b9895") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:38.142428 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:38.142359    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:38.142428 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:38.142396    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret podName:563e6899-0366-4322-a41e-b59d7a4a4b12 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:42.142385005 +0000 UTC m=+10.088869859 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret") pod "global-pull-secret-syncer-68mlb" (UID: "563e6899-0366-4322-a41e-b59d7a4a4b12") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:38.243763 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:38.243029    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twhwc\" (UniqueName: \"kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc\") pod \"network-check-target-wq7g6\" (UID: \"ed17b640-5b2f-4ba7-ac59-1476dfa7ce22\") " pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:38.243763 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:38.243244    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:38.243763 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:38.243267    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:38.243763 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:38.243281    2561 projected.go:196] Error preparing data for projected volume kube-api-access-twhwc for pod openshift-network-diagnostics/network-check-target-wq7g6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:38.243763 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:38.243347    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc podName:ed17b640-5b2f-4ba7-ac59-1476dfa7ce22 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:42.2433272 +0000 UTC m=+10.189812052 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-twhwc" (UniqueName: "kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc") pod "network-check-target-wq7g6" (UID: "ed17b640-5b2f-4ba7-ac59-1476dfa7ce22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:38.601256 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:38.601181    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:38.601765 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:38.601423    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:02:39.597852 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:39.597796    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:39.597852 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:39.597836    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:39.598164 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:39.597981    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:02:39.598236 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:39.598185    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:02:40.598119 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:40.598076    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:40.598609 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:40.598245    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:02:41.598029 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:41.597980    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:41.598237 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:41.597981    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:41.598638 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:41.598263    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:02:41.598638 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:41.598124    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:02:42.179280 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:42.178622    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret\") pod \"global-pull-secret-syncer-68mlb\" (UID: \"563e6899-0366-4322-a41e-b59d7a4a4b12\") " pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:42.179280 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:42.178721    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs\") pod \"network-metrics-daemon-hntnt\" (UID: \"0b4c98d2-1b87-4f67-8881-b4e5639b9895\") " pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:42.179280 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:42.178828    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:42.179280 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:42.178841    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:42.179280 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:42.178913    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs podName:0b4c98d2-1b87-4f67-8881-b4e5639b9895 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:50.178890096 +0000 UTC m=+18.125374955 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs") pod "network-metrics-daemon-hntnt" (UID: "0b4c98d2-1b87-4f67-8881-b4e5639b9895") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:42.179280 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:42.178934    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret podName:563e6899-0366-4322-a41e-b59d7a4a4b12 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:50.178922813 +0000 UTC m=+18.125407659 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret") pod "global-pull-secret-syncer-68mlb" (UID: "563e6899-0366-4322-a41e-b59d7a4a4b12") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:42.280551 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:42.279712    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twhwc\" (UniqueName: \"kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc\") pod \"network-check-target-wq7g6\" (UID: \"ed17b640-5b2f-4ba7-ac59-1476dfa7ce22\") " pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:42.280551 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:42.279953    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:42.280551 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:42.279979    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:42.280551 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:42.279994    2561 projected.go:196] Error preparing data for projected volume kube-api-access-twhwc for pod openshift-network-diagnostics/network-check-target-wq7g6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:42.280551 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:42.280058    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc podName:ed17b640-5b2f-4ba7-ac59-1476dfa7ce22 nodeName:}" failed. No retries permitted until 2026-07-28 19:02:50.280038456 +0000 UTC m=+18.226523307 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-twhwc" (UniqueName: "kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc") pod "network-check-target-wq7g6" (UID: "ed17b640-5b2f-4ba7-ac59-1476dfa7ce22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:42.599290 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:42.598783    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:42.599290 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:42.598914    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:02:43.598376 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:43.598131    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:43.598376 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:43.598144    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:43.598376 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:43.598292    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:02:43.598376 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:43.598341    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:02:44.598386 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:44.597841    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:44.598386 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:44.598003    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:02:45.597872 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:45.597836    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:45.598073 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:45.597836    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:45.598073 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:45.597957    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:02:45.598073 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:45.598014    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:02:46.598247 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:46.598210    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:46.598689 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:46.598346    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:02:47.597923 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:47.597881    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:47.598120 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:47.597887    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:47.598120 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:47.597992    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:02:47.598120 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:47.598095    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:02:48.598752 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:48.598712    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:48.599182 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:48.598851    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:02:49.597731 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:49.597696    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:49.597954 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:49.597694    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:49.597954 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:49.597841    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:02:49.597954 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:49.597882    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:02:50.240162 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:50.240121    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs\") pod \"network-metrics-daemon-hntnt\" (UID: \"0b4c98d2-1b87-4f67-8881-b4e5639b9895\") " pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:50.240800 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:50.240192    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret\") pod \"global-pull-secret-syncer-68mlb\" (UID: \"563e6899-0366-4322-a41e-b59d7a4a4b12\") " pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:50.240800 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:50.240311    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:50.240800 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:50.240394    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs podName:0b4c98d2-1b87-4f67-8881-b4e5639b9895 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:06.240370348 +0000 UTC m=+34.186855200 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs") pod "network-metrics-daemon-hntnt" (UID: "0b4c98d2-1b87-4f67-8881-b4e5639b9895") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:02:50.240800 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:50.240312    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:50.240800 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:50.240463    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret podName:563e6899-0366-4322-a41e-b59d7a4a4b12 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:06.240450685 +0000 UTC m=+34.186935535 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret") pod "global-pull-secret-syncer-68mlb" (UID: "563e6899-0366-4322-a41e-b59d7a4a4b12") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:02:50.341265 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:50.341221    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twhwc\" (UniqueName: \"kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc\") pod \"network-check-target-wq7g6\" (UID: \"ed17b640-5b2f-4ba7-ac59-1476dfa7ce22\") " pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:50.341451 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:50.341421    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:02:50.341451 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:50.341448    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:02:50.341563 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:50.341463    2561 projected.go:196] Error preparing data for projected volume kube-api-access-twhwc for pod openshift-network-diagnostics/network-check-target-wq7g6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:50.341563 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:50.341544    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc podName:ed17b640-5b2f-4ba7-ac59-1476dfa7ce22 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:06.341523783 +0000 UTC m=+34.288008633 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-twhwc" (UniqueName: "kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc") pod "network-check-target-wq7g6" (UID: "ed17b640-5b2f-4ba7-ac59-1476dfa7ce22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:02:50.598625 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:50.598536    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:50.598794 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:50.598695    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:02:51.597735 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:51.597712    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:51.598062 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:51.597712    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:51.598062 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:51.597839    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:02:51.598062 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:51.597876    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:02:52.598736 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:52.598688    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:52.599252 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:52.598852    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:02:53.598147 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:53.598108    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:53.598147 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:53.598142    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:53.598378 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:53.598248    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:02:53.598444 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:53.598378    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:02:54.598735 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:54.598696    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:54.599233 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:54.598845    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:02:55.598038 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:55.597996    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:55.598185 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:55.597996    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:55.598185 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:55.598124    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:02:55.598185 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:55.598177    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:02:56.598178 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.597945    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:56.598877 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:56.598211    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:02:56.791269 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.790990    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j5pqn" event={"ID":"1bc180f1-52ba-4d7f-a886-025e5a39af95","Type":"ContainerStarted","Data":"a0c80aabe752d6a8a07b7e84b7c7c92564aaeecb1a9b5fb342690339d7951cab"}
Jul 28 19:02:56.792593 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.792563    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lxcmx" event={"ID":"40534a0a-48dc-48a6-bfd0-3d0eb410a4c1","Type":"ContainerStarted","Data":"6ebcf970489cf4ad63048571fa98796d38e0577dbfdd39dd343fa0fb3615b70b"}
Jul 28 19:02:56.797538 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.797506    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rbq99" event={"ID":"3c12d37f-cd16-4f11-9681-eaab31e63548","Type":"ContainerStarted","Data":"fd58d4e5700f4dc2790575c4f9409117f56cca13c2c55a5fbf843574c07c557d"}
Jul 28 19:02:56.799040 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.799013    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tks98" event={"ID":"4fb717f7-11df-4903-9208-4e7d35398eb1","Type":"ContainerStarted","Data":"40a82e6877de09a0f6ae7d2dd96a5560cc6b084df909ef4e5ba1a3f2fab6002b"}
Jul 28 19:02:56.800601 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.800575    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ffk9d" event={"ID":"e45507b7-c87c-44b9-b495-b5d812d57324","Type":"ContainerStarted","Data":"492f99d8211452694f87100ee243b588c554ef4c276f1f87a552a7ba82f60165"}
Jul 28 19:02:56.801956 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.801935    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-70.ec2.internal" event={"ID":"683f92978e5f7131a5a5e68a817885c9","Type":"ContainerStarted","Data":"51cfcdb901c4107babc8c9a5e035936b02c121f40cb4ef1ee27b5bfecca60d93"}
Jul 28 19:02:56.803256 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.803235    2561 generic.go:356] "Generic (PLEG): container finished" podID="eb8f3a29ea7328f96977f4eca0275487" containerID="9fc04a73a1e7ad2c4e79ef9628b8ddafb659e0033148ddbacd4164d90f72b398" exitCode=0
Jul 28 19:02:56.803343 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.803306    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal" event={"ID":"eb8f3a29ea7328f96977f4eca0275487","Type":"ContainerDied","Data":"9fc04a73a1e7ad2c4e79ef9628b8ddafb659e0033148ddbacd4164d90f72b398"}
Jul 28 19:02:56.804563 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.804545    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t" event={"ID":"c540efb2-a917-41b1-8b49-756ea74c1d68","Type":"ContainerStarted","Data":"14ccc0b9e2fcdc1adc02a43cddc03bbc55bc00e212b1e978431c5090d964e9fd"}
Jul 28 19:02:56.805954 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.805933    2561 generic.go:356] "Generic (PLEG): container finished" podID="83181e12-1e53-4972-8b1b-6ab591b30cf7" containerID="3276fbd482da8872bfcf1a201da7b230a093de0f7a0849398bda18dcb267a753" exitCode=0
Jul 28 19:02:56.806042 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.805993    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25rm8" event={"ID":"83181e12-1e53-4972-8b1b-6ab591b30cf7","Type":"ContainerDied","Data":"3276fbd482da8872bfcf1a201da7b230a093de0f7a0849398bda18dcb267a753"}
Jul 28 19:02:56.807562 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.807501    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-j5pqn" podStartSLOduration=2.783302178 podStartE2EDuration="23.807462157s" totalImagesPullingTime="21.024159979s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:56.807049479 +0000 UTC m=+24.753534348" watchObservedRunningTime="2026-07-28 19:02:56.807462157 +0000 UTC m=+24.753947026"
Jul 28 19:02:56.809017 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.809000    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:02:56.809314 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.809298    2561 generic.go:356] "Generic (PLEG): container finished" podID="bde3f901-3495-4178-bedc-62e92c8aaecc" containerID="00f8ddc747609f52b8336c5e6d255346b5b404dbb7111159a9a634c55b5e7105" exitCode=1
Jul 28 19:02:56.809380 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.809326    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw" event={"ID":"bde3f901-3495-4178-bedc-62e92c8aaecc","Type":"ContainerStarted","Data":"11d39521df7a3c31ee2bed5870051613040275dadad4793730389207803b2187"}
Jul 28 19:02:56.809414 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.809381    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw" event={"ID":"bde3f901-3495-4178-bedc-62e92c8aaecc","Type":"ContainerStarted","Data":"03933120f4b74f01b1d3834f29a3402c369e4199b6e2dee692c07d798c219f18"}
Jul 28 19:02:56.809443 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.809419    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw" event={"ID":"bde3f901-3495-4178-bedc-62e92c8aaecc","Type":"ContainerStarted","Data":"e5f9f0acd1b3b5236d04247a2308f9d41cfdab370fab8707c0e8fab5503880bb"}
Jul 28 19:02:56.809474 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.809449    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw" event={"ID":"bde3f901-3495-4178-bedc-62e92c8aaecc","Type":"ContainerStarted","Data":"c34cbba2a61d34965c9dd1b6a879f4379b0349662c3f159379f927eff86355b2"}
Jul 28 19:02:56.809527 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.809498    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw" event={"ID":"bde3f901-3495-4178-bedc-62e92c8aaecc","Type":"ContainerDied","Data":"00f8ddc747609f52b8336c5e6d255346b5b404dbb7111159a9a634c55b5e7105"}
Jul 28 19:02:56.809562 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.809533    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw" event={"ID":"bde3f901-3495-4178-bedc-62e92c8aaecc","Type":"ContainerStarted","Data":"103223fd28d16b48b9cd8c8bdf94aff1f28d82c6994ff4edb8709dccb5946322"}
Jul 28 19:02:56.820370 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.820306    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-70.ec2.internal" podStartSLOduration=22.82029092 podStartE2EDuration="22.82029092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:56.820285483 +0000 UTC m=+24.766770350" watchObservedRunningTime="2026-07-28 19:02:56.82029092 +0000 UTC m=+24.766775786"
Jul 28 19:02:56.835187 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.835138    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ffk9d" podStartSLOduration=2.770072691 podStartE2EDuration="23.835121163s" totalImagesPullingTime="21.065048472s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:56.834613485 +0000 UTC m=+24.781098352" watchObservedRunningTime="2026-07-28 19:02:56.835121163 +0000 UTC m=+24.781606031"
Jul 28 19:02:56.851101 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.850972    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tks98" podStartSLOduration=2.8350616840000002 podStartE2EDuration="23.850960491s" totalImagesPullingTime="21.015898807s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:56.850533417 +0000 UTC m=+24.797018306" watchObservedRunningTime="2026-07-28 19:02:56.850960491 +0000 UTC m=+24.797445358"
Jul 28 19:02:56.926505 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.926431    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rbq99" podStartSLOduration=2.8694404589999998 podStartE2EDuration="23.926414067s" totalImagesPullingTime="21.056973608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:56.925864417 +0000 UTC m=+24.872349300" watchObservedRunningTime="2026-07-28 19:02:56.926414067 +0000 UTC m=+24.872898934"
Jul 28 19:02:56.941251 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.941195    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lxcmx" podStartSLOduration=2.912036505 podStartE2EDuration="23.9411762s" totalImagesPullingTime="21.029139695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:56.940642931 +0000 UTC m=+24.887127795" watchObservedRunningTime="2026-07-28 19:02:56.9411762 +0000 UTC m=+24.887661068"
Jul 28 19:02:56.971965 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:56.971929    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lxcmx_40534a0a-48dc-48a6-bfd0-3d0eb410a4c1/konnectivity-agent/0.log" limit=1024
Jul 28 19:02:57.044026 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:57.043983    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lxcmx_40534a0a-48dc-48a6-bfd0-3d0eb410a4c1/konnectivity-agent/0.log" limit=1024
Jul 28 19:02:57.111165 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:57.111077    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lxcmx_40534a0a-48dc-48a6-bfd0-3d0eb410a4c1/konnectivity-agent/0.log" limit=1024
Jul 28 19:02:57.598058 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:57.598020    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:57.598343 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:57.598019    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:57.598343 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:57.598151    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:02:57.598343 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:57.598245    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:02:57.814114 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:57.814079    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nbdhf" event={"ID":"ee2de49f-f6cf-4709-b4b9-f75215cbc196","Type":"ContainerStarted","Data":"6936ceb878a1bf974a60e6f81cd38c63399577194189e17db02226d448c79500"}
Jul 28 19:02:57.853165 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:57.853059    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nbdhf" podStartSLOduration=3.815267251 podStartE2EDuration="24.853040385s" totalImagesPullingTime="21.037773134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:57.852944503 +0000 UTC m=+25.799429370" watchObservedRunningTime="2026-07-28 19:02:57.853040385 +0000 UTC m=+25.799525253"
Jul 28 19:02:57.995608 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:57.995529    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lxcmx_40534a0a-48dc-48a6-bfd0-3d0eb410a4c1/konnectivity-agent/0.log" limit=1024
Jul 28 19:02:58.071383 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:58.071342    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lxcmx"
Jul 28 19:02:58.091060 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:58.091022    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lxcmx_40534a0a-48dc-48a6-bfd0-3d0eb410a4c1/konnectivity-agent/0.log" limit=1024
Jul 28 19:02:58.598502 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:58.598457    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:02:58.598960 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:58.598612    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:02:58.821565 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:58.821534    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:02:58.822027 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:58.821993    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw" event={"ID":"bde3f901-3495-4178-bedc-62e92c8aaecc","Type":"ContainerStarted","Data":"6c2f6bce9f981d867cdab9d0531cd9a582a64427d4c8391cf7a8e197bc453c8e"}
Jul 28 19:02:58.824244 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:58.824208    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal" event={"ID":"eb8f3a29ea7328f96977f4eca0275487","Type":"ContainerStarted","Data":"166a9ec5ed3f8fe552df3ee9f6be2b1d7ec3401b96abd4eefb4ce27bec764706"}
Jul 28 19:02:58.840000 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:58.839910    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-70.ec2.internal" podStartSLOduration=24.839888777 podStartE2EDuration="24.839888777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:02:58.839775498 +0000 UTC m=+26.786260367" watchObservedRunningTime="2026-07-28 19:02:58.839888777 +0000 UTC m=+26.786373664"
Jul 28 19:02:59.254375 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:59.254340    2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 19:02:59.460596 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:59.460494    2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T19:02:59.254363878Z","UUID":"c6ae8aa1-36e4-4557-bb64-055168d56a7d","Handler":null,"Name":"","Endpoint":""}
Jul 28 19:02:59.462120 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:59.462094    2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 19:02:59.462267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:59.462134    2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 19:02:59.598753 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:59.598718    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:02:59.599258 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:59.598723    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:02:59.599258 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:59.598845    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:02:59.599258 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:02:59.598925    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:02:59.829375 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:02:59.829292    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t" event={"ID":"c540efb2-a917-41b1-8b49-756ea74c1d68","Type":"ContainerStarted","Data":"2a2efc57957fb63fc319153f5558e4308d7eac7f0e2e4b2d430191e112b59131"}
Jul 28 19:03:00.399101 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:00.399057    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lxcmx"
Jul 28 19:03:00.399769 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:00.399746    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lxcmx"
Jul 28 19:03:00.598336 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:00.598303    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:03:00.598540 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:00.598452    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:03:00.832737 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:00.832532    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lxcmx"
Jul 28 19:03:00.902362 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:00.902328    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lxcmx_40534a0a-48dc-48a6-bfd0-3d0eb410a4c1/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:01.457735 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:01.457687    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lxcmx_40534a0a-48dc-48a6-bfd0-3d0eb410a4c1/konnectivity-agent/0.log" limit=1024
Jul 28 19:03:01.598345 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:01.598303    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:03:01.598506 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:01.598313    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:03:01.598506 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:01.598438    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:03:01.598506 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:01.598479    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:03:01.836400 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:01.836362    2561 generic.go:356] "Generic (PLEG): container finished" podID="83181e12-1e53-4972-8b1b-6ab591b30cf7" containerID="e114b34a5b10372403b5f7e60e4e4a7516fab2d05787efa447bdacc00d10dd5d" exitCode=0
Jul 28 19:03:01.837239 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:01.836456    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25rm8" event={"ID":"83181e12-1e53-4972-8b1b-6ab591b30cf7","Type":"ContainerDied","Data":"e114b34a5b10372403b5f7e60e4e4a7516fab2d05787efa447bdacc00d10dd5d"}
Jul 28 19:03:01.839971 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:01.839952    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:03:01.840421 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:01.840385    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw" event={"ID":"bde3f901-3495-4178-bedc-62e92c8aaecc","Type":"ContainerStarted","Data":"5dc4934a0e8fca94a4f6665881145eca50aacc0135d020119601e3f6c9a169ae"}
Jul 28 19:03:01.840797 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:01.840768    2561 scope.go:117] "RemoveContainer" containerID="00f8ddc747609f52b8336c5e6d255346b5b404dbb7111159a9a634c55b5e7105"
Jul 28 19:03:02.598992 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.598771    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:03:02.599261 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:02.599040    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:03:02.845413 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.845368    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t" event={"ID":"c540efb2-a917-41b1-8b49-756ea74c1d68","Type":"ContainerStarted","Data":"a3472674906bad6683d748770da46be2fd49ced34c2b8eb1c7c6a686362b3f80"}
Jul 28 19:03:02.851939 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.851820    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:03:02.852186 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.852162    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw" event={"ID":"bde3f901-3495-4178-bedc-62e92c8aaecc","Type":"ContainerStarted","Data":"fcd31323ddfea85e731938102fc8da03850f42da491e15cc32600022a52b2175"}
Jul 28 19:03:02.852458 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.852428    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:03:02.852612 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.852513    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:03:02.852612 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.852560    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:03:02.866663 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.866614    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk59t" podStartSLOduration=3.608867558 podStartE2EDuration="29.866598607s" totalImagesPullingTime="26.257731049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:02.865453503 +0000 UTC m=+30.811938371" watchObservedRunningTime="2026-07-28 19:03:02.866598607 +0000 UTC m=+30.813083474"
Jul 28 19:03:02.868323 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.868293    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:03:02.868433 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.868395    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:03:02.893684 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.893627    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw" podStartSLOduration=8.790648422 podStartE2EDuration="29.893611367s" totalImagesPullingTime="21.102962945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:02.893284925 +0000 UTC m=+30.839769793" watchObservedRunningTime="2026-07-28 19:03:02.893611367 +0000 UTC m=+30.840096276"
Jul 28 19:03:02.911909 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.911867    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-68mlb"]
Jul 28 19:03:02.912076 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.912016    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:03:02.912124 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:02.912097    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:03:02.914900 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.914870    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wq7g6"]
Jul 28 19:03:02.915059 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.915006    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:03:02.915140 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:02.915115    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:03:02.915722 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.915692    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hntnt"]
Jul 28 19:03:02.915861 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:02.915808    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:03:02.915921 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:02.915906    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:03:03.856710 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:03.856678    2561 generic.go:356] "Generic (PLEG): container finished" podID="83181e12-1e53-4972-8b1b-6ab591b30cf7" containerID="e8a4d53d8479d316cb48695f9bb502fda498f6651404ea47281d9904e4aa43a6" exitCode=0
Jul 28 19:03:03.857281 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:03.856762    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25rm8" event={"ID":"83181e12-1e53-4972-8b1b-6ab591b30cf7","Type":"ContainerDied","Data":"e8a4d53d8479d316cb48695f9bb502fda498f6651404ea47281d9904e4aa43a6"}
Jul 28 19:03:04.598697 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:04.598656    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:03:04.598697 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:04.598656    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:03:04.598697 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:04.598676    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:03:04.598943 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:04.598789    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:03:04.598943 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:04.598915    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:03:04.599019 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:04.598992    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:03:05.863628 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:05.863548    2561 generic.go:356] "Generic (PLEG): container finished" podID="83181e12-1e53-4972-8b1b-6ab591b30cf7" containerID="72fd724e7d7d9e69dd0ea6f702549533d4d6afd066f3bad4bb3a50347ce99b4e" exitCode=0
Jul 28 19:03:05.863628 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:05.863598    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25rm8" event={"ID":"83181e12-1e53-4972-8b1b-6ab591b30cf7","Type":"ContainerDied","Data":"72fd724e7d7d9e69dd0ea6f702549533d4d6afd066f3bad4bb3a50347ce99b4e"}
Jul 28 19:03:06.260749 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:06.260705    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret\") pod \"global-pull-secret-syncer-68mlb\" (UID: \"563e6899-0366-4322-a41e-b59d7a4a4b12\") " pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:03:06.260933 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:06.260786    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs\") pod \"network-metrics-daemon-hntnt\" (UID: \"0b4c98d2-1b87-4f67-8881-b4e5639b9895\") " pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:03:06.260933 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:06.260875    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:03:06.260933 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:06.260885    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:03:06.261029 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:06.260949    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret podName:563e6899-0366-4322-a41e-b59d7a4a4b12 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:38.260930364 +0000 UTC m=+66.207415211 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret") pod "global-pull-secret-syncer-68mlb" (UID: "563e6899-0366-4322-a41e-b59d7a4a4b12") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:03:06.261029 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:06.260977    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs podName:0b4c98d2-1b87-4f67-8881-b4e5639b9895 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:38.260964239 +0000 UTC m=+66.207449084 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs") pod "network-metrics-daemon-hntnt" (UID: "0b4c98d2-1b87-4f67-8881-b4e5639b9895") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:03:06.361779 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:06.361742    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twhwc\" (UniqueName: \"kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc\") pod \"network-check-target-wq7g6\" (UID: \"ed17b640-5b2f-4ba7-ac59-1476dfa7ce22\") " pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:03:06.361970 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:06.361897    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:03:06.361970 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:06.361921    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:03:06.361970 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:06.361933    2561 projected.go:196] Error preparing data for projected volume kube-api-access-twhwc for pod openshift-network-diagnostics/network-check-target-wq7g6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:03:06.362124 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:06.361998    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc podName:ed17b640-5b2f-4ba7-ac59-1476dfa7ce22 nodeName:}" failed. No retries permitted until 2026-07-28 19:03:38.361980932 +0000 UTC m=+66.308465798 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-twhwc" (UniqueName: "kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc") pod "network-check-target-wq7g6" (UID: "ed17b640-5b2f-4ba7-ac59-1476dfa7ce22") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:03:06.598130 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:06.598027    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:03:06.598300 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:06.598028    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:03:06.598300 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:06.598156    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wq7g6" podUID="ed17b640-5b2f-4ba7-ac59-1476dfa7ce22"
Jul 28 19:03:06.598300 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:06.598219    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hntnt" podUID="0b4c98d2-1b87-4f67-8881-b4e5639b9895"
Jul 28 19:03:06.598300 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:06.598027    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:03:06.598300 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:06.598294    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-68mlb" podUID="563e6899-0366-4322-a41e-b59d7a4a4b12"
Jul 28 19:03:08.403137 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.403105    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-70.ec2.internal" event="NodeReady"
Jul 28 19:03:08.403601 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.403297    2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 19:03:08.462074 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.462036    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-glvld"]
Jul 28 19:03:08.464626 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.464579    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:08.465741 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.465714    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-c2fr5"]
Jul 28 19:03:08.467666 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.467640    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c2fr5"
Jul 28 19:03:08.467666 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.467660    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 19:03:08.468214 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.468196    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 19:03:08.468338 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.468286    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d4lpd\""
Jul 28 19:03:08.471062 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.471022    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 19:03:08.471178 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.471139    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 19:03:08.471178 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.471161    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nww2p\""
Jul 28 19:03:08.471296 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.471272    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 19:03:08.481214 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.481089    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-glvld"]
Jul 28 19:03:08.481646 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.481612    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c2fr5"]
Jul 28 19:03:08.580221 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.580183    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-898bp\" (UniqueName: \"kubernetes.io/projected/c619cfcb-63dc-4707-beeb-ceeb823e29a9-kube-api-access-898bp\") pod \"ingress-canary-c2fr5\" (UID: \"c619cfcb-63dc-4707-beeb-ceeb823e29a9\") " pod="openshift-ingress-canary/ingress-canary-c2fr5"
Jul 28 19:03:08.580458 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.580228    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1352efd5-8baf-4196-8e4b-4c1b379eca9f-tmp-dir\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:08.580458 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.580265    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1352efd5-8baf-4196-8e4b-4c1b379eca9f-config-volume\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:08.580458 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.580389    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:08.580458 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.580433    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fltz9\" (UniqueName: \"kubernetes.io/projected/1352efd5-8baf-4196-8e4b-4c1b379eca9f-kube-api-access-fltz9\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:08.580692 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.580461    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c619cfcb-63dc-4707-beeb-ceeb823e29a9-cert\") pod \"ingress-canary-c2fr5\" (UID: \"c619cfcb-63dc-4707-beeb-ceeb823e29a9\") " pod="openshift-ingress-canary/ingress-canary-c2fr5"
Jul 28 19:03:08.598268 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.598231    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:03:08.598452 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.598345    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:03:08.598553 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.598233    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:03:08.601275 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.601249    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 19:03:08.601436 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.601299    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 19:03:08.601719 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.601698    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hx2vn\""
Jul 28 19:03:08.601838 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.601742    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 19:03:08.602001 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.601970    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-snpbv\""
Jul 28 19:03:08.602100 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.602014    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 19:03:08.681748 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.681715    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-898bp\" (UniqueName: \"kubernetes.io/projected/c619cfcb-63dc-4707-beeb-ceeb823e29a9-kube-api-access-898bp\") pod \"ingress-canary-c2fr5\" (UID: \"c619cfcb-63dc-4707-beeb-ceeb823e29a9\") " pod="openshift-ingress-canary/ingress-canary-c2fr5"
Jul 28 19:03:08.681748 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.681755    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1352efd5-8baf-4196-8e4b-4c1b379eca9f-tmp-dir\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:08.681990 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.681782    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1352efd5-8baf-4196-8e4b-4c1b379eca9f-config-volume\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:08.682063 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.682022    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:08.682144 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.682105    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fltz9\" (UniqueName: \"kubernetes.io/projected/1352efd5-8baf-4196-8e4b-4c1b379eca9f-kube-api-access-fltz9\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:08.682199 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.682144    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c619cfcb-63dc-4707-beeb-ceeb823e29a9-cert\") pod \"ingress-canary-c2fr5\" (UID: \"c619cfcb-63dc-4707-beeb-ceeb823e29a9\") " pod="openshift-ingress-canary/ingress-canary-c2fr5"
Jul 28 19:03:08.682199 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:08.682149    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:03:08.682295 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.682209    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1352efd5-8baf-4196-8e4b-4c1b379eca9f-tmp-dir\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:08.682295 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:08.682213    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls podName:1352efd5-8baf-4196-8e4b-4c1b379eca9f nodeName:}" failed. No retries permitted until 2026-07-28 19:03:09.182192385 +0000 UTC m=+37.128677250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls") pod "dns-default-glvld" (UID: "1352efd5-8baf-4196-8e4b-4c1b379eca9f") : secret "dns-default-metrics-tls" not found
Jul 28 19:03:08.682524 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.682504    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1352efd5-8baf-4196-8e4b-4c1b379eca9f-config-volume\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:08.687083 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.687057    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c619cfcb-63dc-4707-beeb-ceeb823e29a9-cert\") pod \"ingress-canary-c2fr5\" (UID: \"c619cfcb-63dc-4707-beeb-ceeb823e29a9\") " pod="openshift-ingress-canary/ingress-canary-c2fr5"
Jul 28 19:03:08.692877 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.692821    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fltz9\" (UniqueName: \"kubernetes.io/projected/1352efd5-8baf-4196-8e4b-4c1b379eca9f-kube-api-access-fltz9\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:08.693018 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.692910    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-898bp\" (UniqueName: \"kubernetes.io/projected/c619cfcb-63dc-4707-beeb-ceeb823e29a9-kube-api-access-898bp\") pod \"ingress-canary-c2fr5\" (UID: \"c619cfcb-63dc-4707-beeb-ceeb823e29a9\") " pod="openshift-ingress-canary/ingress-canary-c2fr5"
Jul 28 19:03:08.784498 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:08.784452    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c2fr5"
Jul 28 19:03:09.185373 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:09.185334    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:09.185582 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:09.185524    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:03:09.185633 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:09.185593    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls podName:1352efd5-8baf-4196-8e4b-4c1b379eca9f nodeName:}" failed. No retries permitted until 2026-07-28 19:03:10.18557665 +0000 UTC m=+38.132061495 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls") pod "dns-default-glvld" (UID: "1352efd5-8baf-4196-8e4b-4c1b379eca9f") : secret "dns-default-metrics-tls" not found
Jul 28 19:03:10.049155 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:10.048897    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c2fr5"]
Jul 28 19:03:10.052720 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:03:10.052682    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc619cfcb_63dc_4707_beeb_ceeb823e29a9.slice/crio-a245c2ed273f5a3550d928644eb1efd93031be9d81c760c2a16d04ef87c2528b WatchSource:0}: Error finding container a245c2ed273f5a3550d928644eb1efd93031be9d81c760c2a16d04ef87c2528b: Status 404 returned error can't find the container with id a245c2ed273f5a3550d928644eb1efd93031be9d81c760c2a16d04ef87c2528b
Jul 28 19:03:10.195494 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:10.195449    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:10.195660 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:10.195597    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:03:10.195697 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:10.195664    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls podName:1352efd5-8baf-4196-8e4b-4c1b379eca9f nodeName:}" failed. No retries permitted until 2026-07-28 19:03:12.195646072 +0000 UTC m=+40.142130923 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls") pod "dns-default-glvld" (UID: "1352efd5-8baf-4196-8e4b-4c1b379eca9f") : secret "dns-default-metrics-tls" not found
Jul 28 19:03:10.878147 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:10.878109    2561 generic.go:356] "Generic (PLEG): container finished" podID="83181e12-1e53-4972-8b1b-6ab591b30cf7" containerID="a9d242125cee974675f771ffa1dfe9d35ff3169265a44349967a34d1ec1a994a" exitCode=0
Jul 28 19:03:10.878319 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:10.878193    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25rm8" event={"ID":"83181e12-1e53-4972-8b1b-6ab591b30cf7","Type":"ContainerDied","Data":"a9d242125cee974675f771ffa1dfe9d35ff3169265a44349967a34d1ec1a994a"}
Jul 28 19:03:10.879209 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:10.879190    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c2fr5" event={"ID":"c619cfcb-63dc-4707-beeb-ceeb823e29a9","Type":"ContainerStarted","Data":"a245c2ed273f5a3550d928644eb1efd93031be9d81c760c2a16d04ef87c2528b"}
Jul 28 19:03:11.884529 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:11.884468    2561 generic.go:356] "Generic (PLEG): container finished" podID="83181e12-1e53-4972-8b1b-6ab591b30cf7" containerID="ff527cc4d7f0beb8ccf0e0a3d7c9ff641c6b0c462b2c26e018d074811b63b509" exitCode=0
Jul 28 19:03:11.884529 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:11.884518    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25rm8" event={"ID":"83181e12-1e53-4972-8b1b-6ab591b30cf7","Type":"ContainerDied","Data":"ff527cc4d7f0beb8ccf0e0a3d7c9ff641c6b0c462b2c26e018d074811b63b509"}
Jul 28 19:03:12.210786 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:12.210747    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:12.210970 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:12.210929    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:03:12.211061 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:12.211008    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls podName:1352efd5-8baf-4196-8e4b-4c1b379eca9f nodeName:}" failed. No retries permitted until 2026-07-28 19:03:16.210986679 +0000 UTC m=+44.157471525 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls") pod "dns-default-glvld" (UID: "1352efd5-8baf-4196-8e4b-4c1b379eca9f") : secret "dns-default-metrics-tls" not found
Jul 28 19:03:12.891357 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:12.891269    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25rm8" event={"ID":"83181e12-1e53-4972-8b1b-6ab591b30cf7","Type":"ContainerStarted","Data":"c95196030677d92dac1514c36792e17d13326202287c7e873e4533a9b408a470"}
Jul 28 19:03:12.922887 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:12.922816    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-25rm8" podStartSLOduration=7.5702031 podStartE2EDuration="39.922793125s" totalImagesPullingTime="32.352590025s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:12.921554043 +0000 UTC m=+40.868038927" watchObservedRunningTime="2026-07-28 19:03:12.922793125 +0000 UTC m=+40.869277987"
Jul 28 19:03:14.897216 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:14.897181    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c2fr5" event={"ID":"c619cfcb-63dc-4707-beeb-ceeb823e29a9","Type":"ContainerStarted","Data":"6850c385688425cc0569ca4ec2e9b13d350fe8e643cb608ab4104c9f634285f3"}
Jul 28 19:03:14.912247 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:14.912189    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-c2fr5" podStartSLOduration=2.570517893 podStartE2EDuration="6.912169332s" totalImagesPullingTime="4.341651439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:14.912031197 +0000 UTC m=+42.858516064" watchObservedRunningTime="2026-07-28 19:03:14.912169332 +0000 UTC m=+42.858654201"
Jul 28 19:03:16.243703 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:16.243656    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:16.244093 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:16.243813    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:03:16.244093 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:16.243883    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls podName:1352efd5-8baf-4196-8e4b-4c1b379eca9f nodeName:}" failed. No retries permitted until 2026-07-28 19:03:24.243863909 +0000 UTC m=+52.190348759 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls") pod "dns-default-glvld" (UID: "1352efd5-8baf-4196-8e4b-4c1b379eca9f") : secret "dns-default-metrics-tls" not found
Jul 28 19:03:23.985754 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:23.985719    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"]
Jul 28 19:03:24.001781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.001747    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-xvkq8"]
Jul 28 19:03:24.001949 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.001876    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"
Jul 28 19:03:24.004864 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.004811    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 28 19:03:24.005031 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.004913    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-scmtd\""
Jul 28 19:03:24.005031 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.004982    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 28 19:03:24.016453 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.016414    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-76b66f6655-gwxld"]
Jul 28 19:03:24.017068 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.016521    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:24.020073 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.020047    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:03:24.020274 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.020243    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 28 19:03:24.020374 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.020282    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 28 19:03:24.020374 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.020243    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-whp4l\""
Jul 28 19:03:24.020374 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.020357    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 28 19:03:24.031560 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.031525    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"]
Jul 28 19:03:24.031712 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.031576    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-76b66f6655-gwxld"]
Jul 28 19:03:24.031712 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.031595    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-xvkq8"]
Jul 28 19:03:24.031712 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.031634    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:24.031805 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.031741    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 28 19:03:24.034207 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.034189    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 28 19:03:24.034441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.034424    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 19:03:24.034639 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.034623    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 28 19:03:24.034744 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.034725    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 28 19:03:24.034804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.034763    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 19:03:24.034860 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.034845    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 28 19:03:24.034914 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.034866    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-k4sxb\""
Jul 28 19:03:24.103102 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.103065    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/56684944-6fb3-4f87-91b7-b3f25ec77d7a-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-4vr96\" (UID: \"56684944-6fb3-4f87-91b7-b3f25ec77d7a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"
Jul 28 19:03:24.103289 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.103133    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4vr96\" (UID: \"56684944-6fb3-4f87-91b7-b3f25ec77d7a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"
Jul 28 19:03:24.203981 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.203945    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd7t6\" (UniqueName: \"kubernetes.io/projected/23ad8ae2-6a98-4bda-988d-ac67c7be5946-kube-api-access-qd7t6\") pod \"console-operator-584dcb6b54-xvkq8\" (UID: \"23ad8ae2-6a98-4bda-988d-ac67c7be5946\") " pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:24.203981 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.203990    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-stats-auth\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:24.204209 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.204054    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4vr96\" (UID: \"56684944-6fb3-4f87-91b7-b3f25ec77d7a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"
Jul 28 19:03:24.204209 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.204095    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:24.204209 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.204115    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23ad8ae2-6a98-4bda-988d-ac67c7be5946-trusted-ca\") pod \"console-operator-584dcb6b54-xvkq8\" (UID: \"23ad8ae2-6a98-4bda-988d-ac67c7be5946\") " pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:24.204209 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.204151    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23ad8ae2-6a98-4bda-988d-ac67c7be5946-serving-cert\") pod \"console-operator-584dcb6b54-xvkq8\" (UID: \"23ad8ae2-6a98-4bda-988d-ac67c7be5946\") " pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:24.204365 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:24.204216    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:03:24.204365 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.204231    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/56684944-6fb3-4f87-91b7-b3f25ec77d7a-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-4vr96\" (UID: \"56684944-6fb3-4f87-91b7-b3f25ec77d7a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"
Jul 28 19:03:24.204365 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.204269    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-default-certificate\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:24.204365 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:24.204293    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert podName:56684944-6fb3-4f87-91b7-b3f25ec77d7a nodeName:}" failed. No retries permitted until 2026-07-28 19:03:24.704271248 +0000 UTC m=+52.650756106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4vr96" (UID: "56684944-6fb3-4f87-91b7-b3f25ec77d7a") : secret "networking-console-plugin-cert" not found
Jul 28 19:03:24.204365 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.204354    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23ad8ae2-6a98-4bda-988d-ac67c7be5946-config\") pod \"console-operator-584dcb6b54-xvkq8\" (UID: \"23ad8ae2-6a98-4bda-988d-ac67c7be5946\") " pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:24.204614 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.204408    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5mj5\" (UniqueName: \"kubernetes.io/projected/91043986-5ac8-4cd3-b699-f7029f419bfc-kube-api-access-p5mj5\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:24.204614 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.204447    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:24.204978 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.204959    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/56684944-6fb3-4f87-91b7-b3f25ec77d7a-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-4vr96\" (UID: \"56684944-6fb3-4f87-91b7-b3f25ec77d7a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"
Jul 28 19:03:24.218096 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.218047    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9"]
Jul 28 19:03:24.244257 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.244169    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9"]
Jul 28 19:03:24.244419 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.244290    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9"
Jul 28 19:03:24.248057 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.248030    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 28 19:03:24.248057 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.248030    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:03:24.248314 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.248301    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 28 19:03:24.248375 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.248320    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 28 19:03:24.248375 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.248338    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-grlqq\""
Jul 28 19:03:24.305757 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.305718    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:24.305757 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.305756    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23ad8ae2-6a98-4bda-988d-ac67c7be5946-trusted-ca\") pod \"console-operator-584dcb6b54-xvkq8\" (UID: \"23ad8ae2-6a98-4bda-988d-ac67c7be5946\") " pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:24.305957 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.305782    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23ad8ae2-6a98-4bda-988d-ac67c7be5946-serving-cert\") pod \"console-operator-584dcb6b54-xvkq8\" (UID: \"23ad8ae2-6a98-4bda-988d-ac67c7be5946\") " pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:24.306006 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:24.305980    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:03:24.306098 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.306066    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:24.306098 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:24.306093    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs podName:91043986-5ac8-4cd3-b699-f7029f419bfc nodeName:}" failed. No retries permitted until 2026-07-28 19:03:24.806064548 +0000 UTC m=+52.752549408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs") pod "router-default-76b66f6655-gwxld" (UID: "91043986-5ac8-4cd3-b699-f7029f419bfc") : secret "router-metrics-certs-default" not found
Jul 28 19:03:24.306190 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.306139    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-default-certificate\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:24.306190 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.306175    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23ad8ae2-6a98-4bda-988d-ac67c7be5946-config\") pod \"console-operator-584dcb6b54-xvkq8\" (UID: \"23ad8ae2-6a98-4bda-988d-ac67c7be5946\") " pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:24.306290 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.306210    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5mj5\" (UniqueName: \"kubernetes.io/projected/91043986-5ac8-4cd3-b699-f7029f419bfc-kube-api-access-p5mj5\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:24.306290 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:24.306255    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:03:24.306390 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:24.306327    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls podName:1352efd5-8baf-4196-8e4b-4c1b379eca9f nodeName:}" failed. No retries permitted until 2026-07-28 19:03:40.306308618 +0000 UTC m=+68.252793484 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls") pod "dns-default-glvld" (UID: "1352efd5-8baf-4196-8e4b-4c1b379eca9f") : secret "dns-default-metrics-tls" not found
Jul 28 19:03:24.306390 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.306363    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:24.306526 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.306400    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd7t6\" (UniqueName: \"kubernetes.io/projected/23ad8ae2-6a98-4bda-988d-ac67c7be5946-kube-api-access-qd7t6\") pod \"console-operator-584dcb6b54-xvkq8\" (UID: \"23ad8ae2-6a98-4bda-988d-ac67c7be5946\") " pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:24.306526 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.306433    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-stats-auth\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:24.306645 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:24.306532    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle podName:91043986-5ac8-4cd3-b699-f7029f419bfc nodeName:}" failed. No retries permitted until 2026-07-28 19:03:24.80650887 +0000 UTC m=+52.752993730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle") pod "router-default-76b66f6655-gwxld" (UID: "91043986-5ac8-4cd3-b699-f7029f419bfc") : configmap references non-existent config key: service-ca.crt
Jul 28 19:03:24.307021 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.306999    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23ad8ae2-6a98-4bda-988d-ac67c7be5946-config\") pod \"console-operator-584dcb6b54-xvkq8\" (UID: \"23ad8ae2-6a98-4bda-988d-ac67c7be5946\") " pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:24.307124 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.307073    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23ad8ae2-6a98-4bda-988d-ac67c7be5946-trusted-ca\") pod \"console-operator-584dcb6b54-xvkq8\" (UID: \"23ad8ae2-6a98-4bda-988d-ac67c7be5946\") " pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:24.310221 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.310193    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23ad8ae2-6a98-4bda-988d-ac67c7be5946-serving-cert\") pod \"console-operator-584dcb6b54-xvkq8\" (UID: \"23ad8ae2-6a98-4bda-988d-ac67c7be5946\") " pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:24.310221 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.310218    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-stats-auth\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:24.310355 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.310220    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-default-certificate\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:24.321198 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.321157    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd7t6\" (UniqueName: \"kubernetes.io/projected/23ad8ae2-6a98-4bda-988d-ac67c7be5946-kube-api-access-qd7t6\") pod \"console-operator-584dcb6b54-xvkq8\" (UID: \"23ad8ae2-6a98-4bda-988d-ac67c7be5946\") " pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:24.321696 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.321678    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5mj5\" (UniqueName: \"kubernetes.io/projected/91043986-5ac8-4cd3-b699-f7029f419bfc-kube-api-access-p5mj5\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:24.327897 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.327865    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:24.407457 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.407417    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a096dfea-16af-4e15-8de6-2ab5a01cdc55-config\") pod \"service-ca-operator-5d6c75f84f-j7fw9\" (UID: \"a096dfea-16af-4e15-8de6-2ab5a01cdc55\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9"
Jul 28 19:03:24.407654 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.407564    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nprjr\" (UniqueName: \"kubernetes.io/projected/a096dfea-16af-4e15-8de6-2ab5a01cdc55-kube-api-access-nprjr\") pod \"service-ca-operator-5d6c75f84f-j7fw9\" (UID: \"a096dfea-16af-4e15-8de6-2ab5a01cdc55\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9"
Jul 28 19:03:24.407654 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.407606    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a096dfea-16af-4e15-8de6-2ab5a01cdc55-serving-cert\") pod \"service-ca-operator-5d6c75f84f-j7fw9\" (UID: \"a096dfea-16af-4e15-8de6-2ab5a01cdc55\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9"
Jul 28 19:03:24.482618 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.482583    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-xvkq8"]
Jul 28 19:03:24.486179 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:03:24.486152    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23ad8ae2_6a98_4bda_988d_ac67c7be5946.slice/crio-8d5eea01511ced70712bdd01adaf1fa78a8ca897fc667e935535f54c2605b7e9 WatchSource:0}: Error finding container 8d5eea01511ced70712bdd01adaf1fa78a8ca897fc667e935535f54c2605b7e9: Status 404 returned error can't find the container with id 8d5eea01511ced70712bdd01adaf1fa78a8ca897fc667e935535f54c2605b7e9
Jul 28 19:03:24.508864 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.508763    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nprjr\" (UniqueName: \"kubernetes.io/projected/a096dfea-16af-4e15-8de6-2ab5a01cdc55-kube-api-access-nprjr\") pod \"service-ca-operator-5d6c75f84f-j7fw9\" (UID: \"a096dfea-16af-4e15-8de6-2ab5a01cdc55\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9"
Jul 28 19:03:24.508864 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.508822    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a096dfea-16af-4e15-8de6-2ab5a01cdc55-serving-cert\") pod \"service-ca-operator-5d6c75f84f-j7fw9\" (UID: \"a096dfea-16af-4e15-8de6-2ab5a01cdc55\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9"
Jul 28 19:03:24.509105 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.508913    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a096dfea-16af-4e15-8de6-2ab5a01cdc55-config\") pod \"service-ca-operator-5d6c75f84f-j7fw9\" (UID: \"a096dfea-16af-4e15-8de6-2ab5a01cdc55\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9"
Jul 28 19:03:24.509518 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.509469    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a096dfea-16af-4e15-8de6-2ab5a01cdc55-config\") pod \"service-ca-operator-5d6c75f84f-j7fw9\" (UID: \"a096dfea-16af-4e15-8de6-2ab5a01cdc55\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9"
Jul 28 19:03:24.511145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.511124    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a096dfea-16af-4e15-8de6-2ab5a01cdc55-serving-cert\") pod \"service-ca-operator-5d6c75f84f-j7fw9\" (UID: \"a096dfea-16af-4e15-8de6-2ab5a01cdc55\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9"
Jul 28 19:03:24.517336 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.517313    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nprjr\" (UniqueName: \"kubernetes.io/projected/a096dfea-16af-4e15-8de6-2ab5a01cdc55-kube-api-access-nprjr\") pod \"service-ca-operator-5d6c75f84f-j7fw9\" (UID: \"a096dfea-16af-4e15-8de6-2ab5a01cdc55\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9"
Jul 28 19:03:24.553188 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.553151    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9"
Jul 28 19:03:24.709354 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.709322    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9"]
Jul 28 19:03:24.710607 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.710581    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4vr96\" (UID: \"56684944-6fb3-4f87-91b7-b3f25ec77d7a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"
Jul 28 19:03:24.710738 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:24.710722    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:03:24.710791 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:24.710785    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert podName:56684944-6fb3-4f87-91b7-b3f25ec77d7a nodeName:}" failed. No retries permitted until 2026-07-28 19:03:25.710770597 +0000 UTC m=+53.657255447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4vr96" (UID: "56684944-6fb3-4f87-91b7-b3f25ec77d7a") : secret "networking-console-plugin-cert" not found
Jul 28 19:03:24.723778 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:03:24.723749    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda096dfea_16af_4e15_8de6_2ab5a01cdc55.slice/crio-304352fbe1c5861a8af919197d04fc7143c2be5ec04c84813d110867ec661052 WatchSource:0}: Error finding container 304352fbe1c5861a8af919197d04fc7143c2be5ec04c84813d110867ec661052: Status 404 returned error can't find the container with id 304352fbe1c5861a8af919197d04fc7143c2be5ec04c84813d110867ec661052
Jul 28 19:03:24.811451 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.811356    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:24.811451 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.811424    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:24.811658 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:24.811561    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle podName:91043986-5ac8-4cd3-b699-f7029f419bfc nodeName:}" failed. No retries permitted until 2026-07-28 19:03:25.811542776 +0000 UTC m=+53.758027621 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle") pod "router-default-76b66f6655-gwxld" (UID: "91043986-5ac8-4cd3-b699-f7029f419bfc") : configmap references non-existent config key: service-ca.crt
Jul 28 19:03:24.811658 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:24.811566    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:03:24.811658 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:24.811600    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs podName:91043986-5ac8-4cd3-b699-f7029f419bfc nodeName:}" failed. No retries permitted until 2026-07-28 19:03:25.811593245 +0000 UTC m=+53.758078091 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs") pod "router-default-76b66f6655-gwxld" (UID: "91043986-5ac8-4cd3-b699-f7029f419bfc") : secret "router-metrics-certs-default" not found
Jul 28 19:03:24.922993 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.922958    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8" event={"ID":"23ad8ae2-6a98-4bda-988d-ac67c7be5946","Type":"ContainerStarted","Data":"8d5eea01511ced70712bdd01adaf1fa78a8ca897fc667e935535f54c2605b7e9"}
Jul 28 19:03:24.924017 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:24.923993    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9" event={"ID":"a096dfea-16af-4e15-8de6-2ab5a01cdc55","Type":"ContainerStarted","Data":"304352fbe1c5861a8af919197d04fc7143c2be5ec04c84813d110867ec661052"}
Jul 28 19:03:25.719588 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:25.719543    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4vr96\" (UID: \"56684944-6fb3-4f87-91b7-b3f25ec77d7a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"
Jul 28 19:03:25.720165 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:25.719618    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:03:25.720165 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:25.719704    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert podName:56684944-6fb3-4f87-91b7-b3f25ec77d7a nodeName:}" failed. No retries permitted until 2026-07-28 19:03:27.719681552 +0000 UTC m=+55.666166401 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4vr96" (UID: "56684944-6fb3-4f87-91b7-b3f25ec77d7a") : secret "networking-console-plugin-cert" not found
Jul 28 19:03:25.820228 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:25.820183    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:25.820426 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:25.820276    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:25.820426 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:25.820397    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle podName:91043986-5ac8-4cd3-b699-f7029f419bfc nodeName:}" failed. No retries permitted until 2026-07-28 19:03:27.820372047 +0000 UTC m=+55.766856909 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle") pod "router-default-76b66f6655-gwxld" (UID: "91043986-5ac8-4cd3-b699-f7029f419bfc") : configmap references non-existent config key: service-ca.crt
Jul 28 19:03:25.820587 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:25.820454    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:03:25.820587 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:25.820536    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs podName:91043986-5ac8-4cd3-b699-f7029f419bfc nodeName:}" failed. No retries permitted until 2026-07-28 19:03:27.820518492 +0000 UTC m=+55.767003359 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs") pod "router-default-76b66f6655-gwxld" (UID: "91043986-5ac8-4cd3-b699-f7029f419bfc") : secret "router-metrics-certs-default" not found
Jul 28 19:03:27.734282 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:27.734250    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4vr96\" (UID: \"56684944-6fb3-4f87-91b7-b3f25ec77d7a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"
Jul 28 19:03:27.734651 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:27.734417    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:03:27.734651 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:27.734499    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert podName:56684944-6fb3-4f87-91b7-b3f25ec77d7a nodeName:}" failed. No retries permitted until 2026-07-28 19:03:31.73446502 +0000 UTC m=+59.680949865 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4vr96" (UID: "56684944-6fb3-4f87-91b7-b3f25ec77d7a") : secret "networking-console-plugin-cert" not found
Jul 28 19:03:27.835180 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:27.835141    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:27.835316 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:27.835288    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:03:27.835356 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:27.835291    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:27.835402 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:27.835355    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs podName:91043986-5ac8-4cd3-b699-f7029f419bfc nodeName:}" failed. No retries permitted until 2026-07-28 19:03:31.835337226 +0000 UTC m=+59.781822072 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs") pod "router-default-76b66f6655-gwxld" (UID: "91043986-5ac8-4cd3-b699-f7029f419bfc") : secret "router-metrics-certs-default" not found
Jul 28 19:03:27.835402 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:27.835397    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle podName:91043986-5ac8-4cd3-b699-f7029f419bfc nodeName:}" failed. No retries permitted until 2026-07-28 19:03:31.835380032 +0000 UTC m=+59.781864880 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle") pod "router-default-76b66f6655-gwxld" (UID: "91043986-5ac8-4cd3-b699-f7029f419bfc") : configmap references non-existent config key: service-ca.crt
Jul 28 19:03:27.934922 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:27.934884    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9" event={"ID":"a096dfea-16af-4e15-8de6-2ab5a01cdc55","Type":"ContainerStarted","Data":"72e0d97c696a69b8495ecb0eda1885b7ef769b1cec804b0fea1100a12a328d57"}
Jul 28 19:03:27.936610 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:27.936587    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/0.log"
Jul 28 19:03:27.936735 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:27.936627    2561 generic.go:356] "Generic (PLEG): container finished" podID="23ad8ae2-6a98-4bda-988d-ac67c7be5946" containerID="520a3c0d23b8732236355f665301e05e6980540ea4e5a2099fd95c2445fb716a" exitCode=255
Jul 28 19:03:27.936735 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:27.936659    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8" event={"ID":"23ad8ae2-6a98-4bda-988d-ac67c7be5946","Type":"ContainerDied","Data":"520a3c0d23b8732236355f665301e05e6980540ea4e5a2099fd95c2445fb716a"}
Jul 28 19:03:27.936929 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:27.936913    2561 scope.go:117] "RemoveContainer" containerID="520a3c0d23b8732236355f665301e05e6980540ea4e5a2099fd95c2445fb716a"
Jul 28 19:03:27.952740 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:27.952696    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9" podStartSLOduration=0.924426508 podStartE2EDuration="3.952679534s" totalImagesPullingTime="3.028253026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:27.951895622 +0000 UTC m=+55.898380490" watchObservedRunningTime="2026-07-28 19:03:27.952679534 +0000 UTC m=+55.899164401"
Jul 28 19:03:28.462229 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.462195    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tg8vp"]
Jul 28 19:03:28.465244 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.465221    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:28.468143 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.468106    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-26hct\""
Jul 28 19:03:28.468143 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.468134    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 19:03:28.469086 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.469063    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 19:03:28.469597 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.469573    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 19:03:28.469842 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.469825    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 19:03:28.495866 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.495828    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tg8vp"]
Jul 28 19:03:28.539557 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.539517    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7bf5170-55ae-4c7d-860e-1275f3f63dce-data-volume\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:28.539557 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.539558    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:28.539835 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.539584    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:28.539835 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.539642    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e7bf5170-55ae-4c7d-860e-1275f3f63dce-crio-socket\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:28.539835 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.539720    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slgzg\" (UniqueName: \"kubernetes.io/projected/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-api-access-slgzg\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:28.640841 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.640796    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgzg\" (UniqueName: \"kubernetes.io/projected/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-api-access-slgzg\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:28.641039 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.640887    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7bf5170-55ae-4c7d-860e-1275f3f63dce-data-volume\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:28.641039 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.640906    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:28.641039 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.640932    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:28.641039 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.640956    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e7bf5170-55ae-4c7d-860e-1275f3f63dce-crio-socket\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:28.641236 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.641049    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e7bf5170-55ae-4c7d-860e-1275f3f63dce-crio-socket\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:28.641236 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:28.641056    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:03:28.641236 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:28.641116    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:28.641236 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:28.641130    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm podName:e7bf5170-55ae-4c7d-860e-1275f3f63dce nodeName:}" failed. No retries permitted until 2026-07-28 19:03:29.141109137 +0000 UTC m=+57.087593987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tg8vp" (UID: "e7bf5170-55ae-4c7d-860e-1275f3f63dce") : configmap "kube-rbac-proxy" not found
Jul 28 19:03:28.641236 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:28.641181    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls podName:e7bf5170-55ae-4c7d-860e-1275f3f63dce nodeName:}" failed. No retries permitted until 2026-07-28 19:03:29.141161094 +0000 UTC m=+57.087645967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tg8vp" (UID: "e7bf5170-55ae-4c7d-860e-1275f3f63dce") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:28.641396 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.641286    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7bf5170-55ae-4c7d-860e-1275f3f63dce-data-volume\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:28.650076 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.650037    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slgzg\" (UniqueName: \"kubernetes.io/projected/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-api-access-slgzg\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:28.941290 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.941212    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/1.log"
Jul 28 19:03:28.941757 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.941678    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/0.log"
Jul 28 19:03:28.941757 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.941725    2561 generic.go:356] "Generic (PLEG): container finished" podID="23ad8ae2-6a98-4bda-988d-ac67c7be5946" containerID="14d0bbe3a4c40e5efb89d0ace670642722d4ae8c484c5b08991fb781786b2bbc" exitCode=255
Jul 28 19:03:28.941846 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.941820    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8" event={"ID":"23ad8ae2-6a98-4bda-988d-ac67c7be5946","Type":"ContainerDied","Data":"14d0bbe3a4c40e5efb89d0ace670642722d4ae8c484c5b08991fb781786b2bbc"}
Jul 28 19:03:28.941906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.941893    2561 scope.go:117] "RemoveContainer" containerID="520a3c0d23b8732236355f665301e05e6980540ea4e5a2099fd95c2445fb716a"
Jul 28 19:03:28.942143 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:28.942124    2561 scope.go:117] "RemoveContainer" containerID="14d0bbe3a4c40e5efb89d0ace670642722d4ae8c484c5b08991fb781786b2bbc"
Jul 28 19:03:28.942347 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:28.942318    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-xvkq8_openshift-console-operator(23ad8ae2-6a98-4bda-988d-ac67c7be5946)\"" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8" podUID="23ad8ae2-6a98-4bda-988d-ac67c7be5946"
Jul 28 19:03:29.145222 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:29.145172    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:29.145222 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:29.145223    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:29.145497 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:29.145308    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:03:29.145497 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:29.145340    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:29.145497 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:29.145384    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm podName:e7bf5170-55ae-4c7d-860e-1275f3f63dce nodeName:}" failed. No retries permitted until 2026-07-28 19:03:30.145366777 +0000 UTC m=+58.091851623 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tg8vp" (UID: "e7bf5170-55ae-4c7d-860e-1275f3f63dce") : configmap "kube-rbac-proxy" not found
Jul 28 19:03:29.145497 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:29.145400    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls podName:e7bf5170-55ae-4c7d-860e-1275f3f63dce nodeName:}" failed. No retries permitted until 2026-07-28 19:03:30.145393294 +0000 UTC m=+58.091878140 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tg8vp" (UID: "e7bf5170-55ae-4c7d-860e-1275f3f63dce") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:29.946095 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:29.946068    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/1.log"
Jul 28 19:03:29.946578 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:29.946415    2561 scope.go:117] "RemoveContainer" containerID="14d0bbe3a4c40e5efb89d0ace670642722d4ae8c484c5b08991fb781786b2bbc"
Jul 28 19:03:29.946622 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:29.946583    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-xvkq8_openshift-console-operator(23ad8ae2-6a98-4bda-988d-ac67c7be5946)\"" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8" podUID="23ad8ae2-6a98-4bda-988d-ac67c7be5946"
Jul 28 19:03:30.154222 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:30.154175    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:30.154222 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:30.154222    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:30.154454 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:30.154302    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:03:30.154454 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:30.154333    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:30.154454 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:30.154375    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls podName:e7bf5170-55ae-4c7d-860e-1275f3f63dce nodeName:}" failed. No retries permitted until 2026-07-28 19:03:32.154358028 +0000 UTC m=+60.100842875 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tg8vp" (UID: "e7bf5170-55ae-4c7d-860e-1275f3f63dce") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:30.154454 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:30.154392    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm podName:e7bf5170-55ae-4c7d-860e-1275f3f63dce nodeName:}" failed. No retries permitted until 2026-07-28 19:03:32.154383454 +0000 UTC m=+60.100868301 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tg8vp" (UID: "e7bf5170-55ae-4c7d-860e-1275f3f63dce") : configmap "kube-rbac-proxy" not found
Jul 28 19:03:30.664412 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:30.664383    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j5pqn_1bc180f1-52ba-4d7f-a886-025e5a39af95/dns-node-resolver/0.log"
Jul 28 19:03:31.664656 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:31.664627    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ffk9d_e45507b7-c87c-44b9-b495-b5d812d57324/node-ca/0.log"
Jul 28 19:03:31.768541 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:31.768477    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4vr96\" (UID: \"56684944-6fb3-4f87-91b7-b3f25ec77d7a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"
Jul 28 19:03:31.768709 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:31.768626    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:03:31.768709 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:31.768693    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert podName:56684944-6fb3-4f87-91b7-b3f25ec77d7a nodeName:}" failed. No retries permitted until 2026-07-28 19:03:39.76867702 +0000 UTC m=+67.715161867 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4vr96" (UID: "56684944-6fb3-4f87-91b7-b3f25ec77d7a") : secret "networking-console-plugin-cert" not found
Jul 28 19:03:31.869117 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:31.869071    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:31.869282 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:31.869147    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:31.869282 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:31.869228    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:03:31.869282 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:31.869248    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle podName:91043986-5ac8-4cd3-b699-f7029f419bfc nodeName:}" failed. No retries permitted until 2026-07-28 19:03:39.869231431 +0000 UTC m=+67.815716280 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle") pod "router-default-76b66f6655-gwxld" (UID: "91043986-5ac8-4cd3-b699-f7029f419bfc") : configmap references non-existent config key: service-ca.crt
Jul 28 19:03:31.869282 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:31.869276    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs podName:91043986-5ac8-4cd3-b699-f7029f419bfc nodeName:}" failed. No retries permitted until 2026-07-28 19:03:39.869265355 +0000 UTC m=+67.815750201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs") pod "router-default-76b66f6655-gwxld" (UID: "91043986-5ac8-4cd3-b699-f7029f419bfc") : secret "router-metrics-certs-default" not found
Jul 28 19:03:32.171968 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:32.171929    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:32.171968 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:32.171973    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:32.172194 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:32.172064    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:03:32.172194 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:32.172075    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:32.172194 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:32.172148    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm podName:e7bf5170-55ae-4c7d-860e-1275f3f63dce nodeName:}" failed. No retries permitted until 2026-07-28 19:03:36.172128314 +0000 UTC m=+64.118613183 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tg8vp" (UID: "e7bf5170-55ae-4c7d-860e-1275f3f63dce") : configmap "kube-rbac-proxy" not found
Jul 28 19:03:32.172194 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:32.172176    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls podName:e7bf5170-55ae-4c7d-860e-1275f3f63dce nodeName:}" failed. No retries permitted until 2026-07-28 19:03:36.17216391 +0000 UTC m=+64.118648764 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tg8vp" (UID: "e7bf5170-55ae-4c7d-860e-1275f3f63dce") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:32.264443 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:32.264414    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c2fr5_c619cfcb-63dc-4707-beeb-ceeb823e29a9/serve-healthcheck-canary/0.log"
Jul 28 19:03:34.328227 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:34.328179    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:34.328679 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:34.328244    2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:34.328679 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:34.328641    2561 scope.go:117] "RemoveContainer" containerID="14d0bbe3a4c40e5efb89d0ace670642722d4ae8c484c5b08991fb781786b2bbc"
Jul 28 19:03:34.328829 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:34.328807    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-xvkq8_openshift-console-operator(23ad8ae2-6a98-4bda-988d-ac67c7be5946)\"" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8" podUID="23ad8ae2-6a98-4bda-988d-ac67c7be5946"
Jul 28 19:03:34.876456 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:34.876426    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wcdpw"
Jul 28 19:03:36.206114 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:36.206063    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:36.206114 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:36.206110    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:36.206580 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:36.206216    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:03:36.206580 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:36.206279    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm podName:e7bf5170-55ae-4c7d-860e-1275f3f63dce nodeName:}" failed. No retries permitted until 2026-07-28 19:03:44.20626593 +0000 UTC m=+72.152750779 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tg8vp" (UID: "e7bf5170-55ae-4c7d-860e-1275f3f63dce") : configmap "kube-rbac-proxy" not found
Jul 28 19:03:36.206580 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:36.206310    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:36.206580 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:36.206382    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls podName:e7bf5170-55ae-4c7d-860e-1275f3f63dce nodeName:}" failed. No retries permitted until 2026-07-28 19:03:44.206369456 +0000 UTC m=+72.152854302 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tg8vp" (UID: "e7bf5170-55ae-4c7d-860e-1275f3f63dce") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:38.325934 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:38.325884    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret\") pod \"global-pull-secret-syncer-68mlb\" (UID: \"563e6899-0366-4322-a41e-b59d7a4a4b12\") " pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:03:38.326401 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:38.325981    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs\") pod \"network-metrics-daemon-hntnt\" (UID: \"0b4c98d2-1b87-4f67-8881-b4e5639b9895\") " pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:03:38.328453 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:38.328432    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 19:03:38.328514 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:38.328432    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 19:03:38.336156 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:38.336128    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 19:03:38.336259 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:38.336215    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs podName:0b4c98d2-1b87-4f67-8881-b4e5639b9895 nodeName:}" failed. No retries permitted until 2026-07-28 19:04:42.336190753 +0000 UTC m=+130.282675616 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs") pod "network-metrics-daemon-hntnt" (UID: "0b4c98d2-1b87-4f67-8881-b4e5639b9895") : secret "metrics-daemon-secret" not found
Jul 28 19:03:38.338404 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:38.338370    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/563e6899-0366-4322-a41e-b59d7a4a4b12-original-pull-secret\") pod \"global-pull-secret-syncer-68mlb\" (UID: \"563e6899-0366-4322-a41e-b59d7a4a4b12\") " pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:03:38.426304 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:38.426262    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twhwc\" (UniqueName: \"kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc\") pod \"network-check-target-wq7g6\" (UID: \"ed17b640-5b2f-4ba7-ac59-1476dfa7ce22\") " pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:03:38.428983 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:38.428962    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 19:03:38.439395 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:38.439363    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 19:03:38.449831 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:38.449796    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twhwc\" (UniqueName: \"kubernetes.io/projected/ed17b640-5b2f-4ba7-ac59-1476dfa7ce22-kube-api-access-twhwc\") pod \"network-check-target-wq7g6\" (UID: \"ed17b640-5b2f-4ba7-ac59-1476dfa7ce22\") " pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:03:38.612821 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:38.612726    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hx2vn\""
Jul 28 19:03:38.618099 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:38.618073    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-68mlb"
Jul 28 19:03:38.620214 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:38.620192    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:03:38.766927 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:38.766892    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-68mlb"]
Jul 28 19:03:38.770141 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:03:38.770103    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod563e6899_0366_4322_a41e_b59d7a4a4b12.slice/crio-bce4f276523655a8b9403d9af88263057e9136864347d692e3a4c6c26de7fe35 WatchSource:0}: Error finding container bce4f276523655a8b9403d9af88263057e9136864347d692e3a4c6c26de7fe35: Status 404 returned error can't find the container with id bce4f276523655a8b9403d9af88263057e9136864347d692e3a4c6c26de7fe35
Jul 28 19:03:38.783770 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:38.783735    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wq7g6"]
Jul 28 19:03:38.787021 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:03:38.786995    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded17b640_5b2f_4ba7_ac59_1476dfa7ce22.slice/crio-720d93559e27b0267c885c1bd79f950937201f992aa0718c2b965e448f05ec0e WatchSource:0}: Error finding container 720d93559e27b0267c885c1bd79f950937201f992aa0718c2b965e448f05ec0e: Status 404 returned error can't find the container with id 720d93559e27b0267c885c1bd79f950937201f992aa0718c2b965e448f05ec0e
Jul 28 19:03:38.969142 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:38.969109    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-68mlb" event={"ID":"563e6899-0366-4322-a41e-b59d7a4a4b12","Type":"ContainerStarted","Data":"bce4f276523655a8b9403d9af88263057e9136864347d692e3a4c6c26de7fe35"}
Jul 28 19:03:38.970257 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:38.970229    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wq7g6" event={"ID":"ed17b640-5b2f-4ba7-ac59-1476dfa7ce22","Type":"ContainerStarted","Data":"720d93559e27b0267c885c1bd79f950937201f992aa0718c2b965e448f05ec0e"}
Jul 28 19:03:39.839328 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:39.839273    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4vr96\" (UID: \"56684944-6fb3-4f87-91b7-b3f25ec77d7a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"
Jul 28 19:03:39.839877 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:39.839436    2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:03:39.839877 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:39.839536    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert podName:56684944-6fb3-4f87-91b7-b3f25ec77d7a nodeName:}" failed. No retries permitted until 2026-07-28 19:03:55.839514077 +0000 UTC m=+83.785998938 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4vr96" (UID: "56684944-6fb3-4f87-91b7-b3f25ec77d7a") : secret "networking-console-plugin-cert" not found
Jul 28 19:03:39.940280 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:39.940203    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:39.940515 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:39.940448    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:39.940650 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:39.940626    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle podName:91043986-5ac8-4cd3-b699-f7029f419bfc nodeName:}" failed. No retries permitted until 2026-07-28 19:03:55.940608268 +0000 UTC m=+83.887093115 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle") pod "router-default-76b66f6655-gwxld" (UID: "91043986-5ac8-4cd3-b699-f7029f419bfc") : configmap references non-existent config key: service-ca.crt
Jul 28 19:03:39.943874 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:39.943841    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91043986-5ac8-4cd3-b699-f7029f419bfc-metrics-certs\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:40.344316 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:40.344277    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:40.347500 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:40.347416    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1352efd5-8baf-4196-8e4b-4c1b379eca9f-metrics-tls\") pod \"dns-default-glvld\" (UID: \"1352efd5-8baf-4196-8e4b-4c1b379eca9f\") " pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:40.580996 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:40.580962    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d4lpd\""
Jul 28 19:03:40.588472 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:40.588437    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:40.743191 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:40.743097    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-glvld"]
Jul 28 19:03:40.747973 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:03:40.747942    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1352efd5_8baf_4196_8e4b_4c1b379eca9f.slice/crio-9f8ca4ede3f1737ce8da48fb55d6d16097869e89f6a2fcd3817f247d81e813b1 WatchSource:0}: Error finding container 9f8ca4ede3f1737ce8da48fb55d6d16097869e89f6a2fcd3817f247d81e813b1: Status 404 returned error can't find the container with id 9f8ca4ede3f1737ce8da48fb55d6d16097869e89f6a2fcd3817f247d81e813b1
Jul 28 19:03:40.978268 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:40.978190    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-glvld" event={"ID":"1352efd5-8baf-4196-8e4b-4c1b379eca9f","Type":"ContainerStarted","Data":"9f8ca4ede3f1737ce8da48fb55d6d16097869e89f6a2fcd3817f247d81e813b1"}
Jul 28 19:03:44.279606 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:44.279552    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:44.279606 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:44.279601    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:44.280086 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:44.279760    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:44.280086 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:44.279828    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls podName:e7bf5170-55ae-4c7d-860e-1275f3f63dce nodeName:}" failed. No retries permitted until 2026-07-28 19:04:00.27980978 +0000 UTC m=+88.226294628 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tg8vp" (UID: "e7bf5170-55ae-4c7d-860e-1275f3f63dce") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:03:44.280199 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:44.280180    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7bf5170-55ae-4c7d-860e-1275f3f63dce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:03:44.992718 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:44.992676    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-68mlb" event={"ID":"563e6899-0366-4322-a41e-b59d7a4a4b12","Type":"ContainerStarted","Data":"aaff62eccb8535b0a29e5df843d8457b423233585c52a86b703868e2021061a8"}
Jul 28 19:03:44.994475 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:44.994440    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wq7g6" event={"ID":"ed17b640-5b2f-4ba7-ac59-1476dfa7ce22","Type":"ContainerStarted","Data":"4e8970c6dc0b93a9441b6f4b93c41c1468c572b739e4b59511263d578e7a2fc6"}
Jul 28 19:03:44.994661 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:44.994544    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:03:44.996786 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:44.996758    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-glvld" event={"ID":"1352efd5-8baf-4196-8e4b-4c1b379eca9f","Type":"ContainerStarted","Data":"cad002ec952c7505b8b92107b4668a5bd51fcfe91c8d0e84e7b2f6483a79d5e6"}
Jul 28 19:03:44.996935 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:44.996806    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-glvld" event={"ID":"1352efd5-8baf-4196-8e4b-4c1b379eca9f","Type":"ContainerStarted","Data":"c413a637fa33e748c275f7f89735ce3855243b0d81a97665ca56419d64090404"}
Jul 28 19:03:44.996935 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:44.996881    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:45.009374 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:45.009319    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-68mlb" podStartSLOduration=66.363150205 podStartE2EDuration="1m12.009298313s" totalImagesPullingTime="5.646148108s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:45.00865268 +0000 UTC m=+72.955137587" watchObservedRunningTime="2026-07-28 19:03:45.009298313 +0000 UTC m=+72.955783182"
Jul 28 19:03:45.028246 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:45.028191    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-glvld" podStartSLOduration=32.997386806 podStartE2EDuration="37.028168504s" totalImagesPullingTime="4.030781698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:45.027120613 +0000 UTC m=+72.973605503" watchObservedRunningTime="2026-07-28 19:03:45.028168504 +0000 UTC m=+72.974653381"
Jul 28 19:03:45.043238 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:45.043132    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wq7g6" podStartSLOduration=66.414081691 podStartE2EDuration="1m12.043109465s" totalImagesPullingTime="5.629027774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:45.042924205 +0000 UTC m=+72.989409074" watchObservedRunningTime="2026-07-28 19:03:45.043109465 +0000 UTC m=+72.989594312"
Jul 28 19:03:48.599094 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:48.599051    2561 scope.go:117] "RemoveContainer" containerID="14d0bbe3a4c40e5efb89d0ace670642722d4ae8c484c5b08991fb781786b2bbc"
Jul 28 19:03:49.011925 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:49.011893    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:03:49.012368 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:49.012350    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/1.log"
Jul 28 19:03:49.012442 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:49.012386    2561 generic.go:356] "Generic (PLEG): container finished" podID="23ad8ae2-6a98-4bda-988d-ac67c7be5946" containerID="535903521d6fd6e04195da537c18140fb47b4180e37c8f48e9f580c27d0ef419" exitCode=255
Jul 28 19:03:49.012442 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:49.012425    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8" event={"ID":"23ad8ae2-6a98-4bda-988d-ac67c7be5946","Type":"ContainerDied","Data":"535903521d6fd6e04195da537c18140fb47b4180e37c8f48e9f580c27d0ef419"}
Jul 28 19:03:49.012540 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:49.012477    2561 scope.go:117] "RemoveContainer" containerID="14d0bbe3a4c40e5efb89d0ace670642722d4ae8c484c5b08991fb781786b2bbc"
Jul 28 19:03:49.012832 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:49.012811    2561 scope.go:117] "RemoveContainer" containerID="535903521d6fd6e04195da537c18140fb47b4180e37c8f48e9f580c27d0ef419"
Jul 28 19:03:49.013022 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:49.013003    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-xvkq8_openshift-console-operator(23ad8ae2-6a98-4bda-988d-ac67c7be5946)\"" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8" podUID="23ad8ae2-6a98-4bda-988d-ac67c7be5946"
Jul 28 19:03:50.017593 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:50.017566    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:03:54.328366 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:54.328226    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:54.328366 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:54.328287    2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:03:54.328847 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:54.328592    2561 scope.go:117] "RemoveContainer" containerID="535903521d6fd6e04195da537c18140fb47b4180e37c8f48e9f580c27d0ef419"
Jul 28 19:03:54.328847 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:03:54.328763    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-xvkq8_openshift-console-operator(23ad8ae2-6a98-4bda-988d-ac67c7be5946)\"" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8" podUID="23ad8ae2-6a98-4bda-988d-ac67c7be5946"
Jul 28 19:03:54.921413 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:54.921372    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5ff85868b4-m56kt"]
Jul 28 19:03:54.923612 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:54.923593    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5ff85868b4-m56kt"
Jul 28 19:03:54.925967 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:54.925939    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 28 19:03:54.926882 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:54.926864    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 19:03:54.927004 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:54.926939    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 19:03:54.927067 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:54.927053    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 19:03:54.927129 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:54.927053    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-sznsc\""
Jul 28 19:03:54.936498 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:54.936450    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5ff85868b4-m56kt"]
Jul 28 19:03:55.001587 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.001557    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-glvld"
Jul 28 19:03:55.026811 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.026759    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7468fc8f9d-zxtzw"]
Jul 28 19:03:55.029331 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.029299    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.031830 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.031790    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 19:03:55.032010 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.031790    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jcrsr\""
Jul 28 19:03:55.032010 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.031916    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 19:03:55.032124 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.032025    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 19:03:55.037759 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.037732    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 19:03:55.044710 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.044673    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7468fc8f9d-zxtzw"]
Jul 28 19:03:55.068010 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.067978    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1159dd85-c6ef-4dc5-9c0a-54ab8cab08d7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5ff85868b4-m56kt\" (UID: \"1159dd85-c6ef-4dc5-9c0a-54ab8cab08d7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5ff85868b4-m56kt"
Jul 28 19:03:55.068205 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.068036    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmhfj\" (UniqueName: \"kubernetes.io/projected/1159dd85-c6ef-4dc5-9c0a-54ab8cab08d7-kube-api-access-wmhfj\") pod \"managed-serviceaccount-addon-agent-5ff85868b4-m56kt\" (UID: \"1159dd85-c6ef-4dc5-9c0a-54ab8cab08d7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5ff85868b4-m56kt"
Jul 28 19:03:55.169307 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.169269    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0335142c-a57c-4d34-bea7-8c022dba103c-ca-trust-extracted\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.169531 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.169324    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0335142c-a57c-4d34-bea7-8c022dba103c-trusted-ca\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.169531 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.169379    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0335142c-a57c-4d34-bea7-8c022dba103c-installation-pull-secrets\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.169531 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.169454    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1159dd85-c6ef-4dc5-9c0a-54ab8cab08d7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5ff85868b4-m56kt\" (UID: \"1159dd85-c6ef-4dc5-9c0a-54ab8cab08d7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5ff85868b4-m56kt"
Jul 28 19:03:55.169711 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.169557    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmhfj\" (UniqueName: \"kubernetes.io/projected/1159dd85-c6ef-4dc5-9c0a-54ab8cab08d7-kube-api-access-wmhfj\") pod \"managed-serviceaccount-addon-agent-5ff85868b4-m56kt\" (UID: \"1159dd85-c6ef-4dc5-9c0a-54ab8cab08d7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5ff85868b4-m56kt"
Jul 28 19:03:55.169711 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.169589    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f7m6\" (UniqueName: \"kubernetes.io/projected/0335142c-a57c-4d34-bea7-8c022dba103c-kube-api-access-7f7m6\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.169711 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.169632    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0335142c-a57c-4d34-bea7-8c022dba103c-registry-tls\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.169711 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.169651    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0335142c-a57c-4d34-bea7-8c022dba103c-bound-sa-token\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.169711 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.169697    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0335142c-a57c-4d34-bea7-8c022dba103c-registry-certificates\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.169953 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.169745    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0335142c-a57c-4d34-bea7-8c022dba103c-image-registry-private-configuration\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.172042 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.171982    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1159dd85-c6ef-4dc5-9c0a-54ab8cab08d7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5ff85868b4-m56kt\" (UID: \"1159dd85-c6ef-4dc5-9c0a-54ab8cab08d7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5ff85868b4-m56kt"
Jul 28 19:03:55.186198 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.186169    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmhfj\" (UniqueName: \"kubernetes.io/projected/1159dd85-c6ef-4dc5-9c0a-54ab8cab08d7-kube-api-access-wmhfj\") pod \"managed-serviceaccount-addon-agent-5ff85868b4-m56kt\" (UID: \"1159dd85-c6ef-4dc5-9c0a-54ab8cab08d7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5ff85868b4-m56kt"
Jul 28 19:03:55.253274 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.253234    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5ff85868b4-m56kt"
Jul 28 19:03:55.270348 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.270311    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f7m6\" (UniqueName: \"kubernetes.io/projected/0335142c-a57c-4d34-bea7-8c022dba103c-kube-api-access-7f7m6\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.270540 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.270368    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0335142c-a57c-4d34-bea7-8c022dba103c-registry-tls\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.270540 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.270383    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0335142c-a57c-4d34-bea7-8c022dba103c-bound-sa-token\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.270540 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.270409    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0335142c-a57c-4d34-bea7-8c022dba103c-registry-certificates\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.270540 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.270443    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0335142c-a57c-4d34-bea7-8c022dba103c-image-registry-private-configuration\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.270540 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.270501    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0335142c-a57c-4d34-bea7-8c022dba103c-ca-trust-extracted\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.270540 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.270541    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0335142c-a57c-4d34-bea7-8c022dba103c-trusted-ca\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.270848 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.270568    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0335142c-a57c-4d34-bea7-8c022dba103c-installation-pull-secrets\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.271108 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.271089    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0335142c-a57c-4d34-bea7-8c022dba103c-ca-trust-extracted\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.271582 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.271547    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0335142c-a57c-4d34-bea7-8c022dba103c-registry-certificates\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.271787 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.271769    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0335142c-a57c-4d34-bea7-8c022dba103c-trusted-ca\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.273142 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.273117    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0335142c-a57c-4d34-bea7-8c022dba103c-image-registry-private-configuration\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.273238 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.273156    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0335142c-a57c-4d34-bea7-8c022dba103c-registry-tls\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.273563 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.273545    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0335142c-a57c-4d34-bea7-8c022dba103c-installation-pull-secrets\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.279628 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.279304    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0335142c-a57c-4d34-bea7-8c022dba103c-bound-sa-token\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.279628 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.279444    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f7m6\" (UniqueName: \"kubernetes.io/projected/0335142c-a57c-4d34-bea7-8c022dba103c-kube-api-access-7f7m6\") pod \"image-registry-7468fc8f9d-zxtzw\" (UID: \"0335142c-a57c-4d34-bea7-8c022dba103c\") " pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.340735 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.340375    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:55.392107 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.392067    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5ff85868b4-m56kt"]
Jul 28 19:03:55.397762 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:03:55.397731    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1159dd85_c6ef_4dc5_9c0a_54ab8cab08d7.slice/crio-fc4d0f6f55e0c18f18b852701d81876d0f3de28f9a6358c21290e80b82ca96ac WatchSource:0}: Error finding container fc4d0f6f55e0c18f18b852701d81876d0f3de28f9a6358c21290e80b82ca96ac: Status 404 returned error can't find the container with id fc4d0f6f55e0c18f18b852701d81876d0f3de28f9a6358c21290e80b82ca96ac
Jul 28 19:03:55.490613 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.490569    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7468fc8f9d-zxtzw"]
Jul 28 19:03:55.496896 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:03:55.496847    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0335142c_a57c_4d34_bea7_8c022dba103c.slice/crio-750bef923aa3aa7dd0731c0e0e80a58abd8bc4b7c2f12da0de319318669b7d0f WatchSource:0}: Error finding container 750bef923aa3aa7dd0731c0e0e80a58abd8bc4b7c2f12da0de319318669b7d0f: Status 404 returned error can't find the container with id 750bef923aa3aa7dd0731c0e0e80a58abd8bc4b7c2f12da0de319318669b7d0f
Jul 28 19:03:55.875889 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.875835    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4vr96\" (UID: \"56684944-6fb3-4f87-91b7-b3f25ec77d7a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"
Jul 28 19:03:55.878432 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.878397    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/56684944-6fb3-4f87-91b7-b3f25ec77d7a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4vr96\" (UID: \"56684944-6fb3-4f87-91b7-b3f25ec77d7a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"
Jul 28 19:03:55.977010 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.976965    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:55.977834 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:55.977805    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91043986-5ac8-4cd3-b699-f7029f419bfc-service-ca-bundle\") pod \"router-default-76b66f6655-gwxld\" (UID: \"91043986-5ac8-4cd3-b699-f7029f419bfc\") " pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:56.041840 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:56.041759    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw" event={"ID":"0335142c-a57c-4d34-bea7-8c022dba103c","Type":"ContainerStarted","Data":"a354c2bf9883a378ad819db6ab694402ae8b785eef608ce168c00464d55e7cf5"}
Jul 28 19:03:56.041840 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:56.041828    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw" event={"ID":"0335142c-a57c-4d34-bea7-8c022dba103c","Type":"ContainerStarted","Data":"750bef923aa3aa7dd0731c0e0e80a58abd8bc4b7c2f12da0de319318669b7d0f"}
Jul 28 19:03:56.042093 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:56.041862    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:03:56.043513 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:56.043464    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5ff85868b4-m56kt" event={"ID":"1159dd85-c6ef-4dc5-9c0a-54ab8cab08d7","Type":"ContainerStarted","Data":"fc4d0f6f55e0c18f18b852701d81876d0f3de28f9a6358c21290e80b82ca96ac"}
Jul 28 19:03:56.062964 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:56.062846    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw" podStartSLOduration=1.062824221 podStartE2EDuration="1.062824221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:56.06169435 +0000 UTC m=+84.008179247" watchObservedRunningTime="2026-07-28 19:03:56.062824221 +0000 UTC m=+84.009309089"
Jul 28 19:03:56.127059 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:56.126681    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-scmtd\""
Jul 28 19:03:56.134906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:56.134776    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"
Jul 28 19:03:56.142843 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:56.142797    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-k4sxb\""
Jul 28 19:03:56.151583 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:56.151311    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:56.315385 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:56.315348    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-4vr96"]
Jul 28 19:03:56.317886 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:03:56.317843    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56684944_6fb3_4f87_91b7_b3f25ec77d7a.slice/crio-8174c15b2c2de9cd84d59f84a00e6272c1cea533cd4d63f167fea9b002144456 WatchSource:0}: Error finding container 8174c15b2c2de9cd84d59f84a00e6272c1cea533cd4d63f167fea9b002144456: Status 404 returned error can't find the container with id 8174c15b2c2de9cd84d59f84a00e6272c1cea533cd4d63f167fea9b002144456
Jul 28 19:03:56.338235 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:56.338198    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-76b66f6655-gwxld"]
Jul 28 19:03:56.342558 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:03:56.342478    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91043986_5ac8_4cd3_b699_f7029f419bfc.slice/crio-6261595fda1de3bd3074f5b798a34a702f8a85f113d6a7c23a443644883722fc WatchSource:0}: Error finding container 6261595fda1de3bd3074f5b798a34a702f8a85f113d6a7c23a443644883722fc: Status 404 returned error can't find the container with id 6261595fda1de3bd3074f5b798a34a702f8a85f113d6a7c23a443644883722fc
Jul 28 19:03:57.049086 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:57.049045    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96" event={"ID":"56684944-6fb3-4f87-91b7-b3f25ec77d7a","Type":"ContainerStarted","Data":"8174c15b2c2de9cd84d59f84a00e6272c1cea533cd4d63f167fea9b002144456"}
Jul 28 19:03:57.050741 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:57.050693    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-76b66f6655-gwxld" event={"ID":"91043986-5ac8-4cd3-b699-f7029f419bfc","Type":"ContainerStarted","Data":"5b30f2e2766f03665f3a4347d5fd5e0091e3b37d33e08000588f3e358d1a46c4"}
Jul 28 19:03:57.050860 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:57.050763    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-76b66f6655-gwxld" event={"ID":"91043986-5ac8-4cd3-b699-f7029f419bfc","Type":"ContainerStarted","Data":"6261595fda1de3bd3074f5b798a34a702f8a85f113d6a7c23a443644883722fc"}
Jul 28 19:03:57.071861 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:57.071805    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-76b66f6655-gwxld" podStartSLOduration=34.071782818 podStartE2EDuration="34.071782818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:57.071320379 +0000 UTC m=+85.017805248" watchObservedRunningTime="2026-07-28 19:03:57.071782818 +0000 UTC m=+85.018267687"
Jul 28 19:03:57.151715 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:57.151670    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:57.154849 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:57.154818    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:58.055560 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:58.055458    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5ff85868b4-m56kt" event={"ID":"1159dd85-c6ef-4dc5-9c0a-54ab8cab08d7","Type":"ContainerStarted","Data":"365e575e6c46af2331f5ee2f6f8b06b78b29e8966b291c441d0525b671100958"}
Jul 28 19:03:58.055973 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:58.055693    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:58.056928 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:58.056909    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-76b66f6655-gwxld"
Jul 28 19:03:58.071912 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:58.071842    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5ff85868b4-m56kt" podStartSLOduration=1.705246391 podStartE2EDuration="4.071822219s" totalImagesPullingTime="2.366575828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:58.07101971 +0000 UTC m=+86.017504579" watchObservedRunningTime="2026-07-28 19:03:58.071822219 +0000 UTC m=+86.018307092"
Jul 28 19:03:59.060205 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:59.060149    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96" event={"ID":"56684944-6fb3-4f87-91b7-b3f25ec77d7a","Type":"ContainerStarted","Data":"e407eed01d058e13b6f61ac0e71ecf11955797643cda622e7ffbc31c4bf608c8"}
Jul 28 19:03:59.077422 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:03:59.077356    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4vr96" podStartSLOduration=33.871486367 podStartE2EDuration="36.077338896s" totalImagesPullingTime="2.205852529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:03:59.076247484 +0000 UTC m=+87.022732379" watchObservedRunningTime="2026-07-28 19:03:59.077338896 +0000 UTC m=+87.023823763"
Jul 28 19:04:00.317370 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:00.317323    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:04:00.319741 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:00.319710    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7bf5170-55ae-4c7d-860e-1275f3f63dce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg8vp\" (UID: \"e7bf5170-55ae-4c7d-860e-1275f3f63dce\") " pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:04:00.578996 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:00.578906    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-26hct\""
Jul 28 19:04:00.587042 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:00.587012    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tg8vp"
Jul 28 19:04:00.736379 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:00.736344    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tg8vp"]
Jul 28 19:04:00.739716 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:04:00.739683    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7bf5170_55ae_4c7d_860e_1275f3f63dce.slice/crio-5b30302cdb552e90ba292f98b65522ae06a67e70bbb6f664fb776efedb472958 WatchSource:0}: Error finding container 5b30302cdb552e90ba292f98b65522ae06a67e70bbb6f664fb776efedb472958: Status 404 returned error can't find the container with id 5b30302cdb552e90ba292f98b65522ae06a67e70bbb6f664fb776efedb472958
Jul 28 19:04:01.068123 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:01.068082    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tg8vp" event={"ID":"e7bf5170-55ae-4c7d-860e-1275f3f63dce","Type":"ContainerStarted","Data":"5b30302cdb552e90ba292f98b65522ae06a67e70bbb6f664fb776efedb472958"}
Jul 28 19:04:06.088808 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:06.088715    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tg8vp" event={"ID":"e7bf5170-55ae-4c7d-860e-1275f3f63dce","Type":"ContainerStarted","Data":"7d58a97077f5174ec7fa000ee7d67a8d468f364c520bd06165d4c12d28d4837b"}
Jul 28 19:04:07.599098 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:07.599063    2561 scope.go:117] "RemoveContainer" containerID="535903521d6fd6e04195da537c18140fb47b4180e37c8f48e9f580c27d0ef419"
Jul 28 19:04:07.599654 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:04:07.599390    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-xvkq8_openshift-console-operator(23ad8ae2-6a98-4bda-988d-ac67c7be5946)\"" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8" podUID="23ad8ae2-6a98-4bda-988d-ac67c7be5946"
Jul 28 19:04:09.808362 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.808322    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"]
Jul 28 19:04:09.821603 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.821535    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"
Jul 28 19:04:09.822609 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.822569    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"]
Jul 28 19:04:09.824718 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.824522    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 19:04:09.824718 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.824646    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-8t9pj\""
Jul 28 19:04:09.825370 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.825274    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 28 19:04:09.825370 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.825343    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 19:04:09.825370 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.825353    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 19:04:09.825732 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.825343    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 28 19:04:09.828107 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.827942    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-l62l7"]
Jul 28 19:04:09.836345 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.836229    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:09.839178 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.839145    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kmxrs\""
Jul 28 19:04:09.839692 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.839662    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 19:04:09.839967 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.839704    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 19:04:09.840044 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.839720    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 19:04:09.849498 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.849457    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"]
Jul 28 19:04:09.853100 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.853073    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:09.855676 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.855609    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-kfmdj\""
Jul 28 19:04:09.855905 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.855880    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 28 19:04:09.855905 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.855879    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 28 19:04:09.856242 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.856227    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 28 19:04:09.869086 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:09.869048    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"]
Jul 28 19:04:10.002098 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002060    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-textfile\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.002098 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002103    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a4d213bf-895d-4904-9052-493dd676de9d-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.002353 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002131    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b504f9a-cac5-44ec-9ce1-f17176709745-sys\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.002353 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002198    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-tls\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.002353 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002243    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbwjz\" (UniqueName: \"kubernetes.io/projected/a4d213bf-895d-4904-9052-493dd676de9d-kube-api-access-qbwjz\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.002353 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002276    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4d213bf-895d-4904-9052-493dd676de9d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.002353 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002337    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4d213bf-895d-4904-9052-493dd676de9d-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.002640 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002393    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b504f9a-cac5-44ec-9ce1-f17176709745-metrics-client-ca\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.002640 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002420    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.002640 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002452    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr87p\" (UniqueName: \"kubernetes.io/projected/8b504f9a-cac5-44ec-9ce1-f17176709745-kube-api-access-fr87p\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.002640 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002503    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b504f9a-cac5-44ec-9ce1-f17176709745-root\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.002640 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002533    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-wtmp\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.002640 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002600    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/522f46e3-a690-493f-9b7f-50668ce33072-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-qzwpn\" (UID: \"522f46e3-a690-493f-9b7f-50668ce33072\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"
Jul 28 19:04:10.002918 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002683    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spknd\" (UniqueName: \"kubernetes.io/projected/522f46e3-a690-493f-9b7f-50668ce33072-kube-api-access-spknd\") pod \"openshift-state-metrics-b8f49f5c6-qzwpn\" (UID: \"522f46e3-a690-493f-9b7f-50668ce33072\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"
Jul 28 19:04:10.002918 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002723    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/522f46e3-a690-493f-9b7f-50668ce33072-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-qzwpn\" (UID: \"522f46e3-a690-493f-9b7f-50668ce33072\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"
Jul 28 19:04:10.002918 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002752    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-accelerators-collector-config\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.002918 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002784    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/522f46e3-a690-493f-9b7f-50668ce33072-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-qzwpn\" (UID: \"522f46e3-a690-493f-9b7f-50668ce33072\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"
Jul 28 19:04:10.002918 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002812    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a4d213bf-895d-4904-9052-493dd676de9d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.002918 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.002843    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4d213bf-895d-4904-9052-493dd676de9d-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.103615 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.103527    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b504f9a-cac5-44ec-9ce1-f17176709745-root\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.103615 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.103566    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-wtmp\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.103615 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.103591    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/522f46e3-a690-493f-9b7f-50668ce33072-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-qzwpn\" (UID: \"522f46e3-a690-493f-9b7f-50668ce33072\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"
Jul 28 19:04:10.103899 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.103644    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spknd\" (UniqueName: \"kubernetes.io/projected/522f46e3-a690-493f-9b7f-50668ce33072-kube-api-access-spknd\") pod \"openshift-state-metrics-b8f49f5c6-qzwpn\" (UID: \"522f46e3-a690-493f-9b7f-50668ce33072\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"
Jul 28 19:04:10.103899 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.103674    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/522f46e3-a690-493f-9b7f-50668ce33072-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-qzwpn\" (UID: \"522f46e3-a690-493f-9b7f-50668ce33072\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"
Jul 28 19:04:10.103899 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.103702    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-accelerators-collector-config\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.103899 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.103669    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b504f9a-cac5-44ec-9ce1-f17176709745-root\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.103899 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.103733    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/522f46e3-a690-493f-9b7f-50668ce33072-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-qzwpn\" (UID: \"522f46e3-a690-493f-9b7f-50668ce33072\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"
Jul 28 19:04:10.103899 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.103748    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-wtmp\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.103899 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.103762    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a4d213bf-895d-4904-9052-493dd676de9d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.103899 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.103812    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4d213bf-895d-4904-9052-493dd676de9d-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.103899 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.103848    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-textfile\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.103899 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.103872    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a4d213bf-895d-4904-9052-493dd676de9d-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.103899 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:04:10.103893    2561 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 28 19:04:10.103899 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.103901    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b504f9a-cac5-44ec-9ce1-f17176709745-sys\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.104524 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.103942    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-tls\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.104524 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:04:10.103959    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/522f46e3-a690-493f-9b7f-50668ce33072-openshift-state-metrics-tls podName:522f46e3-a690-493f-9b7f-50668ce33072 nodeName:}" failed. No retries permitted until 2026-07-28 19:04:10.603938624 +0000 UTC m=+98.550423475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/522f46e3-a690-493f-9b7f-50668ce33072-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-qzwpn" (UID: "522f46e3-a690-493f-9b7f-50668ce33072") : secret "openshift-state-metrics-tls" not found
Jul 28 19:04:10.104524 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.104002    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbwjz\" (UniqueName: \"kubernetes.io/projected/a4d213bf-895d-4904-9052-493dd676de9d-kube-api-access-qbwjz\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.104524 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:04:10.104017    2561 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 19:04:10.104524 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.104041    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4d213bf-895d-4904-9052-493dd676de9d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.104524 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:04:10.104067    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-tls podName:8b504f9a-cac5-44ec-9ce1-f17176709745 nodeName:}" failed. No retries permitted until 2026-07-28 19:04:10.604051676 +0000 UTC m=+98.550536525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-tls") pod "node-exporter-l62l7" (UID: "8b504f9a-cac5-44ec-9ce1-f17176709745") : secret "node-exporter-tls" not found
Jul 28 19:04:10.104524 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.104103    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4d213bf-895d-4904-9052-493dd676de9d-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.104524 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.104143    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b504f9a-cac5-44ec-9ce1-f17176709745-metrics-client-ca\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.104524 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.104171    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.104524 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.104205    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr87p\" (UniqueName: \"kubernetes.io/projected/8b504f9a-cac5-44ec-9ce1-f17176709745-kube-api-access-fr87p\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.104524 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.104405    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-accelerators-collector-config\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.104524 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.104471    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b504f9a-cac5-44ec-9ce1-f17176709745-sys\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.105125 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.104614    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/522f46e3-a690-493f-9b7f-50668ce33072-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-qzwpn\" (UID: \"522f46e3-a690-493f-9b7f-50668ce33072\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"
Jul 28 19:04:10.105125 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.104646    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a4d213bf-895d-4904-9052-493dd676de9d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.105125 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.105026    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-textfile\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.105284 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.105130    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a4d213bf-895d-4904-9052-493dd676de9d-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.105424 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.105372    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4d213bf-895d-4904-9052-493dd676de9d-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.105527 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.105502    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b504f9a-cac5-44ec-9ce1-f17176709745-metrics-client-ca\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.107035 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.107003    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/522f46e3-a690-493f-9b7f-50668ce33072-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-qzwpn\" (UID: \"522f46e3-a690-493f-9b7f-50668ce33072\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"
Jul 28 19:04:10.107369 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.107344    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a4d213bf-895d-4904-9052-493dd676de9d-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.107600 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.107579    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4d213bf-895d-4904-9052-493dd676de9d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.107600 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.107590    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.125882 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.125845    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbwjz\" (UniqueName: \"kubernetes.io/projected/a4d213bf-895d-4904-9052-493dd676de9d-kube-api-access-qbwjz\") pod \"kube-state-metrics-56d9f944b4-dr8wz\" (UID: \"a4d213bf-895d-4904-9052-493dd676de9d\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.126068 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.125969    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr87p\" (UniqueName: \"kubernetes.io/projected/8b504f9a-cac5-44ec-9ce1-f17176709745-kube-api-access-fr87p\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.126068 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.125999    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spknd\" (UniqueName: \"kubernetes.io/projected/522f46e3-a690-493f-9b7f-50668ce33072-kube-api-access-spknd\") pod \"openshift-state-metrics-b8f49f5c6-qzwpn\" (UID: \"522f46e3-a690-493f-9b7f-50668ce33072\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"
Jul 28 19:04:10.163914 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.163866    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"
Jul 28 19:04:10.319062 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.319026    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz"]
Jul 28 19:04:10.322622 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:04:10.322585    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4d213bf_895d_4904_9052_493dd676de9d.slice/crio-7961d2fa9827bb8b5ff699429ebe973e00afbfb68ea169f54c9dce2e60d3e038 WatchSource:0}: Error finding container 7961d2fa9827bb8b5ff699429ebe973e00afbfb68ea169f54c9dce2e60d3e038: Status 404 returned error can't find the container with id 7961d2fa9827bb8b5ff699429ebe973e00afbfb68ea169f54c9dce2e60d3e038
Jul 28 19:04:10.609639 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.609603    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-tls\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.609873 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.609712    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/522f46e3-a690-493f-9b7f-50668ce33072-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-qzwpn\" (UID: \"522f46e3-a690-493f-9b7f-50668ce33072\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"
Jul 28 19:04:10.612111 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.612081    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b504f9a-cac5-44ec-9ce1-f17176709745-node-exporter-tls\") pod \"node-exporter-l62l7\" (UID: \"8b504f9a-cac5-44ec-9ce1-f17176709745\") " pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.612226 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.612177    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/522f46e3-a690-493f-9b7f-50668ce33072-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-qzwpn\" (UID: \"522f46e3-a690-493f-9b7f-50668ce33072\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"
Jul 28 19:04:10.739789 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.739743    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"
Jul 28 19:04:10.750095 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.750069    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l62l7"
Jul 28 19:04:10.760458 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:04:10.760419    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b504f9a_cac5_44ec_9ce1_f17176709745.slice/crio-a4426be560efd6d61eecb5e3093dfe7d045ed486710803a95beec62edf44feef WatchSource:0}: Error finding container a4426be560efd6d61eecb5e3093dfe7d045ed486710803a95beec62edf44feef: Status 404 returned error can't find the container with id a4426be560efd6d61eecb5e3093dfe7d045ed486710803a95beec62edf44feef
Jul 28 19:04:10.898680 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:10.898652    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn"]
Jul 28 19:04:10.901773 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:04:10.901742    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod522f46e3_a690_493f_9b7f_50668ce33072.slice/crio-f163da27a8f0d6349b51f4cde25e4e1c0be0d36801ebde8b56069704e17486da WatchSource:0}: Error finding container f163da27a8f0d6349b51f4cde25e4e1c0be0d36801ebde8b56069704e17486da: Status 404 returned error can't find the container with id f163da27a8f0d6349b51f4cde25e4e1c0be0d36801ebde8b56069704e17486da
Jul 28 19:04:11.110594 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:11.110517    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn" event={"ID":"522f46e3-a690-493f-9b7f-50668ce33072","Type":"ContainerStarted","Data":"2a7cd128a59bcbfe3a2ea278d4863f746602f0385ca0ab2563ef97d4823df88c"}
Jul 28 19:04:11.110758 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:11.110598    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn" event={"ID":"522f46e3-a690-493f-9b7f-50668ce33072","Type":"ContainerStarted","Data":"52ad37c675e9e16e5e3015c1d786149e8182d236dccac558ec42d25fc5002585"}
Jul 28 19:04:11.110758 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:11.110632    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn" event={"ID":"522f46e3-a690-493f-9b7f-50668ce33072","Type":"ContainerStarted","Data":"f163da27a8f0d6349b51f4cde25e4e1c0be0d36801ebde8b56069704e17486da"}
Jul 28 19:04:11.112222 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:11.112132    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l62l7" event={"ID":"8b504f9a-cac5-44ec-9ce1-f17176709745","Type":"ContainerStarted","Data":"a4426be560efd6d61eecb5e3093dfe7d045ed486710803a95beec62edf44feef"}
Jul 28 19:04:11.114155 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:11.114076    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tg8vp" event={"ID":"e7bf5170-55ae-4c7d-860e-1275f3f63dce","Type":"ContainerStarted","Data":"1382841500fde75cd8a5d8adba7effd94827dafa0832af08fdc23cea55b92a9a"}
Jul 28 19:04:11.115794 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:11.115715    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz" event={"ID":"a4d213bf-895d-4904-9052-493dd676de9d","Type":"ContainerStarted","Data":"7961d2fa9827bb8b5ff699429ebe973e00afbfb68ea169f54c9dce2e60d3e038"}
Jul 28 19:04:12.125891 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:12.125846    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz" event={"ID":"a4d213bf-895d-4904-9052-493dd676de9d","Type":"ContainerStarted","Data":"2f961c42a0000732ea0a2be427f47762ac9bd9d5c8bf2332c5d17ec30dd41e01"}
Jul 28 19:04:12.126419 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:12.125929    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz" event={"ID":"a4d213bf-895d-4904-9052-493dd676de9d","Type":"ContainerStarted","Data":"0aec2c566d5e31638c15b9baf7f918ad6b184c12773c0bd04a64383dabd3c9e7"}
Jul 28 19:04:12.126419 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:12.125963    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz" event={"ID":"a4d213bf-895d-4904-9052-493dd676de9d","Type":"ContainerStarted","Data":"cf408ba83e88a41f88ecd6a4ff8e649db8a5ea406cb095b1ba14eb67fe1348cf"}
Jul 28 19:04:12.148906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:12.148792    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-dr8wz" podStartSLOduration=1.831137354 podStartE2EDuration="3.148772818s" totalImagesPullingTime="1.317635464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:12.146868812 +0000 UTC m=+100.093353695" watchObservedRunningTime="2026-07-28 19:04:12.148772818 +0000 UTC m=+100.095257685"
Jul 28 19:04:14.293478 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.293435    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7759958bbb-455qk"]
Jul 28 19:04:14.407348 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.407315    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7759958bbb-455qk"]
Jul 28 19:04:14.407566 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.407439    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.410360 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.410322    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 28 19:04:14.410552 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.410325    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4pksw\""
Jul 28 19:04:14.410730 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.410691    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 19:04:14.410730 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.410699    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-abm292vpbuh87\""
Jul 28 19:04:14.410866 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.410774    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 28 19:04:14.410928 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.410875    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 28 19:04:14.444257 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.444213    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04a819c6-fd34-4174-ac7a-2459c9c96e71-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.444446 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.444316    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/04a819c6-fd34-4174-ac7a-2459c9c96e71-audit-log\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.444446 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.444345    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhkdz\" (UniqueName: \"kubernetes.io/projected/04a819c6-fd34-4174-ac7a-2459c9c96e71-kube-api-access-lhkdz\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.444568 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.444451    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/04a819c6-fd34-4174-ac7a-2459c9c96e71-secret-metrics-server-tls\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.444568 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.444527    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/04a819c6-fd34-4174-ac7a-2459c9c96e71-metrics-server-audit-profiles\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.444568 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.444562    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/04a819c6-fd34-4174-ac7a-2459c9c96e71-client-ca-bundle\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.444711 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.444667    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/04a819c6-fd34-4174-ac7a-2459c9c96e71-secret-metrics-server-client-certs\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.545629 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.545518    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/04a819c6-fd34-4174-ac7a-2459c9c96e71-secret-metrics-server-client-certs\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.545629 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.545582    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04a819c6-fd34-4174-ac7a-2459c9c96e71-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.545866 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.545643    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/04a819c6-fd34-4174-ac7a-2459c9c96e71-audit-log\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.545866 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.545672    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhkdz\" (UniqueName: \"kubernetes.io/projected/04a819c6-fd34-4174-ac7a-2459c9c96e71-kube-api-access-lhkdz\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.545866 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.545720    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/04a819c6-fd34-4174-ac7a-2459c9c96e71-secret-metrics-server-tls\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.546028 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.545889    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/04a819c6-fd34-4174-ac7a-2459c9c96e71-metrics-server-audit-profiles\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.546028 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.545937    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/04a819c6-fd34-4174-ac7a-2459c9c96e71-client-ca-bundle\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.546232 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.546181    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/04a819c6-fd34-4174-ac7a-2459c9c96e71-audit-log\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.546767 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.546744    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04a819c6-fd34-4174-ac7a-2459c9c96e71-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.546931 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.546908    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/04a819c6-fd34-4174-ac7a-2459c9c96e71-metrics-server-audit-profiles\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.548959 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.548937    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/04a819c6-fd34-4174-ac7a-2459c9c96e71-client-ca-bundle\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.549408 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.549388    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/04a819c6-fd34-4174-ac7a-2459c9c96e71-secret-metrics-server-client-certs\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.549517 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.549436    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/04a819c6-fd34-4174-ac7a-2459c9c96e71-secret-metrics-server-tls\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.559614 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.559220    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhkdz\" (UniqueName: \"kubernetes.io/projected/04a819c6-fd34-4174-ac7a-2459c9c96e71-kube-api-access-lhkdz\") pod \"metrics-server-7759958bbb-455qk\" (UID: \"04a819c6-fd34-4174-ac7a-2459c9c96e71\") " pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.625040 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.625005    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv"]
Jul 28 19:04:14.674365 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.674326    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv"]
Jul 28 19:04:14.674595 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.674506    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv"
Jul 28 19:04:14.677444 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.677418    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-lkvlg\""
Jul 28 19:04:14.677607 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.677541    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 28 19:04:14.718501 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.718440    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:14.747694 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.747658    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxt4r\" (UniqueName: \"kubernetes.io/projected/13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a-kube-api-access-nxt4r\") pod \"monitoring-plugin-68b7fc9574-btjkv\" (UID: \"13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv"
Jul 28 19:04:14.747694 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.747694    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-btjkv\" (UID: \"13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv"
Jul 28 19:04:14.848826 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.848741    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxt4r\" (UniqueName: \"kubernetes.io/projected/13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a-kube-api-access-nxt4r\") pod \"monitoring-plugin-68b7fc9574-btjkv\" (UID: \"13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv"
Jul 28 19:04:14.848826 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.848784    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-btjkv\" (UID: \"13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv"
Jul 28 19:04:14.849084 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:04:14.848917    2561 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 28 19:04:14.849084 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:04:14.848979    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a-monitoring-plugin-cert podName:13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a nodeName:}" failed. No retries permitted until 2026-07-28 19:04:15.348961871 +0000 UTC m=+103.295446721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-btjkv" (UID: "13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a") : secret "monitoring-plugin-cert" not found
Jul 28 19:04:14.862850 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:14.862824    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxt4r\" (UniqueName: \"kubernetes.io/projected/13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a-kube-api-access-nxt4r\") pod \"monitoring-plugin-68b7fc9574-btjkv\" (UID: \"13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv"
Jul 28 19:04:15.142784 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:15.142749    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l62l7" event={"ID":"8b504f9a-cac5-44ec-9ce1-f17176709745","Type":"ContainerStarted","Data":"1b3a709c2743554a621bd9307925d150dbcd7547c725e4d7f0fe6f35d2ab811e"}
Jul 28 19:04:15.144735 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:15.144685    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn" event={"ID":"522f46e3-a690-493f-9b7f-50668ce33072","Type":"ContainerStarted","Data":"2ab5ee92601762d424cdac1f9dbb58ad1051da4ec015d7fa929ae1223b7c59f8"}
Jul 28 19:04:15.155668 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:15.155618    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7759958bbb-455qk"]
Jul 28 19:04:15.160306 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:04:15.160254    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04a819c6_fd34_4174_ac7a_2459c9c96e71.slice/crio-6d9d62e910a17693133642044f7a6bb32fe2a2db451648b86dcd059103de955d WatchSource:0}: Error finding container 6d9d62e910a17693133642044f7a6bb32fe2a2db451648b86dcd059103de955d: Status 404 returned error can't find the container with id 6d9d62e910a17693133642044f7a6bb32fe2a2db451648b86dcd059103de955d
Jul 28 19:04:15.196397 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:15.196098    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-qzwpn" podStartSLOduration=2.249564023 podStartE2EDuration="6.196080577s" totalImagesPullingTime="3.946516554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:15.195564934 +0000 UTC m=+103.142049824" watchObservedRunningTime="2026-07-28 19:04:15.196080577 +0000 UTC m=+103.142565445"
Jul 28 19:04:15.353832 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:15.353790    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-btjkv\" (UID: \"13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv"
Jul 28 19:04:15.356331 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:15.356255    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-btjkv\" (UID: \"13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv"
Jul 28 19:04:15.585204 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:15.585159    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv"
Jul 28 19:04:15.752943 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:15.752906    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv"]
Jul 28 19:04:15.756409 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:04:15.756374    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13f9a1c2_3b3a_4e1a_aa99_f4dbec47256a.slice/crio-3bcfb7d921f2d1ec560551979bdffa0dcf496c9dfc446371837128b8c48359b0 WatchSource:0}: Error finding container 3bcfb7d921f2d1ec560551979bdffa0dcf496c9dfc446371837128b8c48359b0: Status 404 returned error can't find the container with id 3bcfb7d921f2d1ec560551979bdffa0dcf496c9dfc446371837128b8c48359b0
Jul 28 19:04:16.002249 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.002221    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wq7g6"
Jul 28 19:04:16.111936 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.111899    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 19:04:16.135025 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.134989    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 19:04:16.135194 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.135158    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.138231 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.138201    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 19:04:16.138231 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.138220    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 19:04:16.138441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.138252    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 19:04:16.138441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.138256    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 19:04:16.138627 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.138608    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 19:04:16.138627 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.138624    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dahsqirpds1fu\""
Jul 28 19:04:16.138744 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.138654    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 19:04:16.138744 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.138627    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 19:04:16.138744 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.138693    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 19:04:16.138744 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.138655    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 19:04:16.138936 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.138772    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-42fxs\""
Jul 28 19:04:16.138936 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.138843    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 19:04:16.139568 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.139550    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 19:04:16.139669 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.139581    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 19:04:16.142349 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.142327    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 19:04:16.145498 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.145467    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 19:04:16.154700 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.154669    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tg8vp" event={"ID":"e7bf5170-55ae-4c7d-860e-1275f3f63dce","Type":"ContainerStarted","Data":"2b06a3837d4aebda5a2b16e0c7a3b8950aa49fafcaa05007eb8eeb01ecbd8e07"}
Jul 28 19:04:16.155773 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.155746    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7759958bbb-455qk" event={"ID":"04a819c6-fd34-4174-ac7a-2459c9c96e71","Type":"ContainerStarted","Data":"6d9d62e910a17693133642044f7a6bb32fe2a2db451648b86dcd059103de955d"}
Jul 28 19:04:16.157278 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.157256    2561 generic.go:356] "Generic (PLEG): container finished" podID="8b504f9a-cac5-44ec-9ce1-f17176709745" containerID="1b3a709c2743554a621bd9307925d150dbcd7547c725e4d7f0fe6f35d2ab811e" exitCode=0
Jul 28 19:04:16.157379 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.157315    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l62l7" event={"ID":"8b504f9a-cac5-44ec-9ce1-f17176709745","Type":"ContainerDied","Data":"1b3a709c2743554a621bd9307925d150dbcd7547c725e4d7f0fe6f35d2ab811e"}
Jul 28 19:04:16.158377 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.158356    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv" event={"ID":"13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a","Type":"ContainerStarted","Data":"3bcfb7d921f2d1ec560551979bdffa0dcf496c9dfc446371837128b8c48359b0"}
Jul 28 19:04:16.161476 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.161453    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.161476 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.161501    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.161736 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.161705    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.161801 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.161770    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.162531 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.161799    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-config\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.162531 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.161862    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-config-out\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.162531 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.161911    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.162531 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.162106    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.162531 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.162241    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.162889 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.162851    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-web-config\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.162945 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.162896    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.162945 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.162938    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.163057 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.163034    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.163104 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.163084    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.163138 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.163112    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.163178 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.163138    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.163213 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.163182    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f72jz\" (UniqueName: \"kubernetes.io/projected/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-kube-api-access-f72jz\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.163252 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.163220    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.163291 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.163262    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.163331 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.163290    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.201664 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.201609    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tg8vp" podStartSLOduration=34.063670831 podStartE2EDuration="48.201588487s" totalImagesPullingTime="14.137917656s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:03:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:16.200933969 +0000 UTC m=+104.147418837" watchObservedRunningTime="2026-07-28 19:04:16.201588487 +0000 UTC m=+104.148073355"
Jul 28 19:04:16.264358 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.264264    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.264358 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.264306    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.264358 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.264336    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.264659 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.264377    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-web-config\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.264659 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.264404    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.264659 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.264424    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.264811 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.264668    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.264811 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.264711    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.264811 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.264756    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.264811 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.264783    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.265022 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.264806    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.265022 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.264848    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f72jz\" (UniqueName: \"kubernetes.io/projected/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-kube-api-access-f72jz\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.265022 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.264890    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.265022 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.264928    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.265022 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.264955    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.265022 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.265015    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.265308 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.265043    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.265308 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.265079    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.265308 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.265112    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.265308 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.265143    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-config\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.265308 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.265174    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-config-out\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.265575 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.265312    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.265575 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.265426    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.265575 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.265518    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.266193 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.266167    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.267365 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.267015    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.267365 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.267328    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.269936 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.269435    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.269936 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.269463    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.270262 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.270216    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.270512 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.270469    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-web-config\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.271230 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.271199    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.271514 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.271444    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-config-out\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.272192 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.271923    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.272192 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.272052    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-config\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.272394 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.272282    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.272394 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.272369    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.272529 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.272453    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.273422 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.273401    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.282273 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.282244    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f72jz\" (UniqueName: \"kubernetes.io/projected/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-kube-api-access-f72jz\") pod \"prometheus-k8s-0\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.448432 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.448379    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:16.735065 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:16.735028    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 19:04:16.740214 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:04:16.740174    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc645e8b_667e_4cc4_9ea4_9c7201dfccc6.slice/crio-a8982c5c0c5c3c115b058888ecd9258072a1f18358d193b893dddfa672c35dcc WatchSource:0}: Error finding container a8982c5c0c5c3c115b058888ecd9258072a1f18358d193b893dddfa672c35dcc: Status 404 returned error can't find the container with id a8982c5c0c5c3c115b058888ecd9258072a1f18358d193b893dddfa672c35dcc
Jul 28 19:04:17.056716 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:17.056685    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7468fc8f9d-zxtzw"
Jul 28 19:04:17.166208 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:17.166168    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l62l7" event={"ID":"8b504f9a-cac5-44ec-9ce1-f17176709745","Type":"ContainerStarted","Data":"739da89ba354b0564619956ae509fe67543acc33b39214f7d5c95e9e284041a3"}
Jul 28 19:04:17.166400 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:17.166248    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l62l7" event={"ID":"8b504f9a-cac5-44ec-9ce1-f17176709745","Type":"ContainerStarted","Data":"3c9b981091971ad2688fac2e50fcc0bb0a3dad95d2b420ff6bb69b8028d519dc"}
Jul 28 19:04:17.167717 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:17.167691    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6","Type":"ContainerStarted","Data":"a8982c5c0c5c3c115b058888ecd9258072a1f18358d193b893dddfa672c35dcc"}
Jul 28 19:04:17.169521 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:17.169470    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7759958bbb-455qk" event={"ID":"04a819c6-fd34-4174-ac7a-2459c9c96e71","Type":"ContainerStarted","Data":"7f7934127ac5030a26952d7f7c1099e1904b8651ec08aa0a40e10b95d2be1431"}
Jul 28 19:04:17.236464 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:17.236393    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-l62l7" podStartSLOduration=3.8958466290000002 podStartE2EDuration="8.236368176s" totalImagesPullingTime="4.235355852s" totalInitContainerRuntime="105.165695ms" isStatefulPod=true podCreationTimestamp="2026-07-28 19:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:17.199136539 +0000 UTC m=+105.145621432" watchObservedRunningTime="2026-07-28 19:04:17.236368176 +0000 UTC m=+105.182853047"
Jul 28 19:04:17.236817 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:17.236783    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7759958bbb-455qk" podStartSLOduration=1.793781025 podStartE2EDuration="3.236772417s" totalImagesPullingTime="1.442991392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:04:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:17.236096117 +0000 UTC m=+105.182581004" watchObservedRunningTime="2026-07-28 19:04:17.236772417 +0000 UTC m=+105.183257286"
Jul 28 19:04:18.174776 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:18.174675    2561 generic.go:356] "Generic (PLEG): container finished" podID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerID="ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e" exitCode=0
Jul 28 19:04:18.175194 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:18.174762    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6","Type":"ContainerDied","Data":"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e"}
Jul 28 19:04:18.176557 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:18.176526    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv" event={"ID":"13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a","Type":"ContainerStarted","Data":"46bf1a88d1b85cfa49d37729d8e0c0daaa55e58361fffab2fae7acd1b441bbb3"}
Jul 28 19:04:18.176864 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:18.176845    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv"
Jul 28 19:04:18.181664 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:18.181637    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv"
Jul 28 19:04:18.221718 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:18.221661    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-btjkv" podStartSLOduration=2.232662971 podStartE2EDuration="4.221640609s" totalImagesPullingTime="1.988977638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:04:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:18.221090194 +0000 UTC m=+106.167575089" watchObservedRunningTime="2026-07-28 19:04:18.221640609 +0000 UTC m=+106.168125476"
Jul 28 19:04:19.599054 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:19.599013    2561 scope.go:117] "RemoveContainer" containerID="535903521d6fd6e04195da537c18140fb47b4180e37c8f48e9f580c27d0ef419"
Jul 28 19:04:20.187261 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:20.187225    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:04:20.187433 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:20.187363    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8" event={"ID":"23ad8ae2-6a98-4bda-988d-ac67c7be5946","Type":"ContainerStarted","Data":"cc6fdd1662cc3f80e5c71031d2c46d053023d6fd3762070cb647333e0862b545"}
Jul 28 19:04:20.187943 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:20.187921    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:04:20.193525 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:20.193477    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8"
Jul 28 19:04:20.218622 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:20.218558    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-xvkq8" podStartSLOduration=53.957840115 podStartE2EDuration="57.21854157s" totalImagesPullingTime="3.260701455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:03:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:20.217040842 +0000 UTC m=+108.163525731" watchObservedRunningTime="2026-07-28 19:04:20.21854157 +0000 UTC m=+108.165026438"
Jul 28 19:04:21.196587 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:21.196538    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6","Type":"ContainerStarted","Data":"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1"}
Jul 28 19:04:21.197086 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:21.196643    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6","Type":"ContainerStarted","Data":"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000"}
Jul 28 19:04:23.209366 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:23.209327    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6","Type":"ContainerStarted","Data":"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e"}
Jul 28 19:04:23.209792 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:23.209438    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6","Type":"ContainerStarted","Data":"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9"}
Jul 28 19:04:23.209792 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:23.209465    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6","Type":"ContainerStarted","Data":"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad"}
Jul 28 19:04:24.216924 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:24.216882    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6","Type":"ContainerStarted","Data":"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8"}
Jul 28 19:04:24.247429 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:24.247363    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.532271892 podStartE2EDuration="8.247338785s" totalImagesPullingTime="5.693111441s" totalInitContainerRuntime="21.955452ms" isStatefulPod=false podCreationTimestamp="2026-07-28 19:04:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:24.246068876 +0000 UTC m=+112.192553788" watchObservedRunningTime="2026-07-28 19:04:24.247338785 +0000 UTC m=+112.193823718"
Jul 28 19:04:26.449164 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:26.449116    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:04:34.719076 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:34.719026    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:34.719470 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:34.719096    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:42.405879 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:42.405846    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs\") pod \"network-metrics-daemon-hntnt\" (UID: \"0b4c98d2-1b87-4f67-8881-b4e5639b9895\") " pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:04:42.408287 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:42.408265    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b4c98d2-1b87-4f67-8881-b4e5639b9895-metrics-certs\") pod \"network-metrics-daemon-hntnt\" (UID: \"0b4c98d2-1b87-4f67-8881-b4e5639b9895\") " pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:04:42.527120 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:42.527089    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-snpbv\""
Jul 28 19:04:42.535439 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:42.535411    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hntnt"
Jul 28 19:04:43.011945 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:43.011908    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hntnt"]
Jul 28 19:04:43.015295 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:04:43.015256    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b4c98d2_1b87_4f67_8881_b4e5639b9895.slice/crio-e9dcecdd2dd1601aeb2f818492b756871394204ca8bd607bd896dd89b56a2f3c WatchSource:0}: Error finding container e9dcecdd2dd1601aeb2f818492b756871394204ca8bd607bd896dd89b56a2f3c: Status 404 returned error can't find the container with id e9dcecdd2dd1601aeb2f818492b756871394204ca8bd607bd896dd89b56a2f3c
Jul 28 19:04:43.293102 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:43.293061    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hntnt" event={"ID":"0b4c98d2-1b87-4f67-8881-b4e5639b9895","Type":"ContainerStarted","Data":"e9dcecdd2dd1601aeb2f818492b756871394204ca8bd607bd896dd89b56a2f3c"}
Jul 28 19:04:45.304843 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:45.304809    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hntnt" event={"ID":"0b4c98d2-1b87-4f67-8881-b4e5639b9895","Type":"ContainerStarted","Data":"229c4702899cc5cd1abf323968527e3b0e10eeba790baecd85ca6998627f198a"}
Jul 28 19:04:46.311199 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:46.311159    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hntnt" event={"ID":"0b4c98d2-1b87-4f67-8881-b4e5639b9895","Type":"ContainerStarted","Data":"16f704a2596891f434e2b1823cf38c65da5d73e2d56df663421fc63de55dcaa1"}
Jul 28 19:04:46.329886 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:46.329824    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hntnt" podStartSLOduration=131.346277532 podStartE2EDuration="2m13.329803126s" totalImagesPullingTime="1.983525594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:04:46.328063882 +0000 UTC m=+134.274548769" watchObservedRunningTime="2026-07-28 19:04:46.329803126 +0000 UTC m=+134.276288042"
Jul 28 19:04:49.737020 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:49.736984    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76b66f6655-gwxld_91043986-5ac8-4cd3-b699-f7029f419bfc/router/0.log"
Jul 28 19:04:49.750652 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:49.750620    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c2fr5_c619cfcb-63dc-4707-beeb-ceeb823e29a9/serve-healthcheck-canary/0.log"
Jul 28 19:04:53.341631 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:53.341529    2561 generic.go:356] "Generic (PLEG): container finished" podID="a096dfea-16af-4e15-8de6-2ab5a01cdc55" containerID="72e0d97c696a69b8495ecb0eda1885b7ef769b1cec804b0fea1100a12a328d57" exitCode=0
Jul 28 19:04:53.341631 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:53.341607    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9" event={"ID":"a096dfea-16af-4e15-8de6-2ab5a01cdc55","Type":"ContainerDied","Data":"72e0d97c696a69b8495ecb0eda1885b7ef769b1cec804b0fea1100a12a328d57"}
Jul 28 19:04:53.342147 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:53.342010    2561 scope.go:117] "RemoveContainer" containerID="72e0d97c696a69b8495ecb0eda1885b7ef769b1cec804b0fea1100a12a328d57"
Jul 28 19:04:54.347255 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:54.347220    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-j7fw9" event={"ID":"a096dfea-16af-4e15-8de6-2ab5a01cdc55","Type":"ContainerStarted","Data":"c9a87bd1babc830d93c64e9c11371d8ea428b55eb5a82f0be6a20d47a2892ed2"}
Jul 28 19:04:54.724369 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:54.724338    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:04:54.728343 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:04:54.728311    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7759958bbb-455qk"
Jul 28 19:05:16.449247 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:16.449204    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:16.466097 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:16.466067    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:17.457758 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:17.457730    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:34.491317 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.491222    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 19:05:34.491892 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.491687    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="kube-rbac-proxy-thanos" containerID="cri-o://ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8" gracePeriod=600
Jul 28 19:05:34.491892 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.491679    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="prometheus" containerID="cri-o://5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000" gracePeriod=600
Jul 28 19:05:34.491892 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.491740    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="thanos-sidecar" containerID="cri-o://e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad" gracePeriod=600
Jul 28 19:05:34.491892 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.491767    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="config-reloader" containerID="cri-o://4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1" gracePeriod=600
Jul 28 19:05:34.491892 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.491735    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="kube-rbac-proxy-web" containerID="cri-o://3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9" gracePeriod=600
Jul 28 19:05:34.491892 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.491800    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="kube-rbac-proxy" containerID="cri-o://df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e" gracePeriod=600
Jul 28 19:05:34.736964 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.736941    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:34.869632 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.869547    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-2\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.869632 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.869594    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-1\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.869632 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.869613    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-serving-certs-ca-bundle\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.869906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.869647    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-web-config\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.869906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.869678    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-kube-rbac-proxy\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.869906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.869722    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-trusted-ca-bundle\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.869906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.869757    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-thanos-prometheus-http-client-file\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.869906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.869808    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-db\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.869906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.869836    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.869906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.869877    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-metrics-client-ca\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.869906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.869906    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.870281 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.869938    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-tls-assets\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.870281 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.869990    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:05:34.870281 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.870010    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-grpc-tls\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.870281 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.870079    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-kubelet-serving-ca-bundle\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.870281 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.870076    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:05:34.870281 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.870110    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f72jz\" (UniqueName: \"kubernetes.io/projected/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-kube-api-access-f72jz\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.870281 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.870136    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-tls\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.870281 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.870165    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-config\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.870281 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.870170    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:05:34.870281 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.870189    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-0\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.870281 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.870217    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-config-out\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.870281 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.870242    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-metrics-client-certs\") pod \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\" (UID: \"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6\") "
Jul 28 19:05:34.870910 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.870411    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:05:34.870910 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.870570    2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-trusted-ca-bundle\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.870910 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.870592    2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.870910 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.870606    2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.870910 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.870622    2561 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.871148 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.871086    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:05:34.871474 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.871423    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:05:34.871721 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.871699    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:05:34.872702 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.872676    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:34.872921 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.872883    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:05:34.873012 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.872991    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:34.874535 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.874510    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:34.874627 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.874546    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:34.874627 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.874600    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:34.874627 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.874624    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-config-out" (OuterVolumeSpecName: "config-out") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:05:34.874861 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.874834    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-kube-api-access-f72jz" (OuterVolumeSpecName: "kube-api-access-f72jz") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "kube-api-access-f72jz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:05:34.875031 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.875007    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:05:34.875121 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.875005    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:34.875121 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.875054    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:34.876118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.876099    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-config" (OuterVolumeSpecName: "config") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:34.883756 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.883734    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-web-config" (OuterVolumeSpecName: "web-config") pod "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" (UID: "dc645e8b-667e-4cc4-9ea4-9c7201dfccc6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:05:34.972028 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.971995    2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-web-config\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.972028 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.972025    2561 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-kube-rbac-proxy\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.972028 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.972035    2561 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-thanos-prometheus-http-client-file\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.972271 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.972047    2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-db\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.972271 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.972057    2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.972271 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.972068    2561 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-metrics-client-ca\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.972271 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.972079    2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.972271 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.972087    2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-tls-assets\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.972271 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.972095    2561 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-grpc-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.972271 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.972103    2561 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.972271 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.972111    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f72jz\" (UniqueName: \"kubernetes.io/projected/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-kube-api-access-f72jz\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.972271 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.972120    2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-prometheus-k8s-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.972271 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.972128    2561 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-config\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.972271 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.972136    2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.972271 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.972146    2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-config-out\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:34.972271 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:34.972154    2561 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6-secret-metrics-client-certs\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:05:35.514101 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.514066    2561 generic.go:356] "Generic (PLEG): container finished" podID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerID="ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8" exitCode=0
Jul 28 19:05:35.514101 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.514096    2561 generic.go:356] "Generic (PLEG): container finished" podID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerID="df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e" exitCode=0
Jul 28 19:05:35.514101 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.514106    2561 generic.go:356] "Generic (PLEG): container finished" podID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerID="3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9" exitCode=0
Jul 28 19:05:35.514101 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.514116    2561 generic.go:356] "Generic (PLEG): container finished" podID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerID="e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad" exitCode=0
Jul 28 19:05:35.514101 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.514123    2561 generic.go:356] "Generic (PLEG): container finished" podID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerID="4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1" exitCode=0
Jul 28 19:05:35.514762 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.514132    2561 generic.go:356] "Generic (PLEG): container finished" podID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerID="5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000" exitCode=0
Jul 28 19:05:35.514762 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.514157    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6","Type":"ContainerDied","Data":"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8"}
Jul 28 19:05:35.514762 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.514171    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.514762 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.514253    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6","Type":"ContainerDied","Data":"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e"}
Jul 28 19:05:35.514762 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.514299    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6","Type":"ContainerDied","Data":"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9"}
Jul 28 19:05:35.514762 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.514341    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6","Type":"ContainerDied","Data":"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad"}
Jul 28 19:05:35.514762 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.514352    2561 scope.go:117] "RemoveContainer" containerID="ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8"
Jul 28 19:05:35.514762 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.514383    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6","Type":"ContainerDied","Data":"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1"}
Jul 28 19:05:35.514762 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.514425    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6","Type":"ContainerDied","Data":"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000"}
Jul 28 19:05:35.514762 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.514470    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc645e8b-667e-4cc4-9ea4-9c7201dfccc6","Type":"ContainerDied","Data":"a8982c5c0c5c3c115b058888ecd9258072a1f18358d193b893dddfa672c35dcc"}
Jul 28 19:05:35.522607 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.522584    2561 scope.go:117] "RemoveContainer" containerID="df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e"
Jul 28 19:05:35.530187 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.530165    2561 scope.go:117] "RemoveContainer" containerID="3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9"
Jul 28 19:05:35.537465 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.537436    2561 scope.go:117] "RemoveContainer" containerID="e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad"
Jul 28 19:05:35.540337 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.540314    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 19:05:35.545973 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.545953    2561 scope.go:117] "RemoveContainer" containerID="4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1"
Jul 28 19:05:35.546052 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.545970    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 19:05:35.552998 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.552979    2561 scope.go:117] "RemoveContainer" containerID="5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000"
Jul 28 19:05:35.560760 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.560740    2561 scope.go:117] "RemoveContainer" containerID="ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e"
Jul 28 19:05:35.567923 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.567903    2561 scope.go:117] "RemoveContainer" containerID="ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8"
Jul 28 19:05:35.568208 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:05:35.568182    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8\": container with ID starting with ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8 not found: ID does not exist" containerID="ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8"
Jul 28 19:05:35.568285 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.568221    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8"} err="failed to get container status \"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8\": rpc error: code = NotFound desc = could not find container \"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8\": container with ID starting with ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8 not found: ID does not exist"
Jul 28 19:05:35.568348 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.568293    2561 scope.go:117] "RemoveContainer" containerID="df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e"
Jul 28 19:05:35.568614 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:05:35.568593    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e\": container with ID starting with df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e not found: ID does not exist" containerID="df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e"
Jul 28 19:05:35.568687 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.568621    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e"} err="failed to get container status \"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e\": rpc error: code = NotFound desc = could not find container \"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e\": container with ID starting with df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e not found: ID does not exist"
Jul 28 19:05:35.568687 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.568639    2561 scope.go:117] "RemoveContainer" containerID="3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9"
Jul 28 19:05:35.568925 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:05:35.568906    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9\": container with ID starting with 3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9 not found: ID does not exist" containerID="3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9"
Jul 28 19:05:35.568965 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.568931    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9"} err="failed to get container status \"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9\": rpc error: code = NotFound desc = could not find container \"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9\": container with ID starting with 3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9 not found: ID does not exist"
Jul 28 19:05:35.568965 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.568948    2561 scope.go:117] "RemoveContainer" containerID="e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad"
Jul 28 19:05:35.569192 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:05:35.569175    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad\": container with ID starting with e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad not found: ID does not exist" containerID="e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad"
Jul 28 19:05:35.569261 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.569200    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad"} err="failed to get container status \"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad\": rpc error: code = NotFound desc = could not find container \"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad\": container with ID starting with e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad not found: ID does not exist"
Jul 28 19:05:35.569261 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.569219    2561 scope.go:117] "RemoveContainer" containerID="4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1"
Jul 28 19:05:35.569541 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:05:35.569479    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1\": container with ID starting with 4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1 not found: ID does not exist" containerID="4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1"
Jul 28 19:05:35.569603 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.569547    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1"} err="failed to get container status \"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1\": rpc error: code = NotFound desc = could not find container \"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1\": container with ID starting with 4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1 not found: ID does not exist"
Jul 28 19:05:35.569603 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.569563    2561 scope.go:117] "RemoveContainer" containerID="5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000"
Jul 28 19:05:35.569809 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:05:35.569790    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000\": container with ID starting with 5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000 not found: ID does not exist" containerID="5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000"
Jul 28 19:05:35.569853 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.569814    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000"} err="failed to get container status \"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000\": rpc error: code = NotFound desc = could not find container \"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000\": container with ID starting with 5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000 not found: ID does not exist"
Jul 28 19:05:35.569853 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.569828    2561 scope.go:117] "RemoveContainer" containerID="ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e"
Jul 28 19:05:35.570050 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:05:35.570033    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e\": container with ID starting with ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e not found: ID does not exist" containerID="ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e"
Jul 28 19:05:35.570110 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.570058    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e"} err="failed to get container status \"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e\": rpc error: code = NotFound desc = could not find container \"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e\": container with ID starting with ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e not found: ID does not exist"
Jul 28 19:05:35.570110 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.570077    2561 scope.go:117] "RemoveContainer" containerID="ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8"
Jul 28 19:05:35.570302 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.570286    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8"} err="failed to get container status \"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8\": rpc error: code = NotFound desc = could not find container \"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8\": container with ID starting with ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8 not found: ID does not exist"
Jul 28 19:05:35.570353 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.570305    2561 scope.go:117] "RemoveContainer" containerID="df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e"
Jul 28 19:05:35.570773 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.570752    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e"} err="failed to get container status \"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e\": rpc error: code = NotFound desc = could not find container \"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e\": container with ID starting with df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e not found: ID does not exist"
Jul 28 19:05:35.570855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.570774    2561 scope.go:117] "RemoveContainer" containerID="3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9"
Jul 28 19:05:35.570987 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.570969    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9"} err="failed to get container status \"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9\": rpc error: code = NotFound desc = could not find container \"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9\": container with ID starting with 3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9 not found: ID does not exist"
Jul 28 19:05:35.571032 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.570989    2561 scope.go:117] "RemoveContainer" containerID="e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad"
Jul 28 19:05:35.571196 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.571173    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad"} err="failed to get container status \"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad\": rpc error: code = NotFound desc = could not find container \"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad\": container with ID starting with e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad not found: ID does not exist"
Jul 28 19:05:35.571240 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.571199    2561 scope.go:117] "RemoveContainer" containerID="4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1"
Jul 28 19:05:35.571383 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.571363    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1"} err="failed to get container status \"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1\": rpc error: code = NotFound desc = could not find container \"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1\": container with ID starting with 4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1 not found: ID does not exist"
Jul 28 19:05:35.571427 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.571385    2561 scope.go:117] "RemoveContainer" containerID="5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000"
Jul 28 19:05:35.571603 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.571585    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000"} err="failed to get container status \"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000\": rpc error: code = NotFound desc = could not find container \"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000\": container with ID starting with 5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000 not found: ID does not exist"
Jul 28 19:05:35.571662 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.571603    2561 scope.go:117] "RemoveContainer" containerID="ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e"
Jul 28 19:05:35.571795 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.571780    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e"} err="failed to get container status \"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e\": rpc error: code = NotFound desc = could not find container \"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e\": container with ID starting with ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e not found: ID does not exist"
Jul 28 19:05:35.571835 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.571795    2561 scope.go:117] "RemoveContainer" containerID="ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8"
Jul 28 19:05:35.572000 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.571986    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8"} err="failed to get container status \"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8\": rpc error: code = NotFound desc = could not find container \"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8\": container with ID starting with ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8 not found: ID does not exist"
Jul 28 19:05:35.572045 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.572000    2561 scope.go:117] "RemoveContainer" containerID="df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e"
Jul 28 19:05:35.572188 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.572173    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e"} err="failed to get container status \"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e\": rpc error: code = NotFound desc = could not find container \"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e\": container with ID starting with df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e not found: ID does not exist"
Jul 28 19:05:35.572236 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.572188    2561 scope.go:117] "RemoveContainer" containerID="3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9"
Jul 28 19:05:35.572388 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.572369    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9"} err="failed to get container status \"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9\": rpc error: code = NotFound desc = could not find container \"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9\": container with ID starting with 3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9 not found: ID does not exist"
Jul 28 19:05:35.572454 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.572390    2561 scope.go:117] "RemoveContainer" containerID="e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad"
Jul 28 19:05:35.572605 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.572587    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad"} err="failed to get container status \"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad\": rpc error: code = NotFound desc = could not find container \"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad\": container with ID starting with e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad not found: ID does not exist"
Jul 28 19:05:35.572649 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.572607    2561 scope.go:117] "RemoveContainer" containerID="4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1"
Jul 28 19:05:35.572832 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.572815    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1"} err="failed to get container status \"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1\": rpc error: code = NotFound desc = could not find container \"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1\": container with ID starting with 4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1 not found: ID does not exist"
Jul 28 19:05:35.572897 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.572833    2561 scope.go:117] "RemoveContainer" containerID="5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000"
Jul 28 19:05:35.573017 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.573001    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000"} err="failed to get container status \"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000\": rpc error: code = NotFound desc = could not find container \"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000\": container with ID starting with 5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000 not found: ID does not exist"
Jul 28 19:05:35.573081 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.573018    2561 scope.go:117] "RemoveContainer" containerID="ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e"
Jul 28 19:05:35.573341 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.573316    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e"} err="failed to get container status \"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e\": rpc error: code = NotFound desc = could not find container \"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e\": container with ID starting with ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e not found: ID does not exist"
Jul 28 19:05:35.573429 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.573342    2561 scope.go:117] "RemoveContainer" containerID="ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8"
Jul 28 19:05:35.573720 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.573694    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8"} err="failed to get container status \"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8\": rpc error: code = NotFound desc = could not find container \"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8\": container with ID starting with ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8 not found: ID does not exist"
Jul 28 19:05:35.573830 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.573818    2561 scope.go:117] "RemoveContainer" containerID="df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e"
Jul 28 19:05:35.574241 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.574221    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e"} err="failed to get container status \"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e\": rpc error: code = NotFound desc = could not find container \"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e\": container with ID starting with df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e not found: ID does not exist"
Jul 28 19:05:35.574241 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.574241    2561 scope.go:117] "RemoveContainer" containerID="3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9"
Jul 28 19:05:35.574564 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.574543    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9"} err="failed to get container status \"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9\": rpc error: code = NotFound desc = could not find container \"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9\": container with ID starting with 3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9 not found: ID does not exist"
Jul 28 19:05:35.574564 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.574564    2561 scope.go:117] "RemoveContainer" containerID="e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad"
Jul 28 19:05:35.574820 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.574798    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad"} err="failed to get container status \"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad\": rpc error: code = NotFound desc = could not find container \"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad\": container with ID starting with e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad not found: ID does not exist"
Jul 28 19:05:35.574820 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.574820    2561 scope.go:117] "RemoveContainer" containerID="4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1"
Jul 28 19:05:35.575062 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575037    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1"} err="failed to get container status \"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1\": rpc error: code = NotFound desc = could not find container \"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1\": container with ID starting with 4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1 not found: ID does not exist"
Jul 28 19:05:35.575135 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575063    2561 scope.go:117] "RemoveContainer" containerID="5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000"
Jul 28 19:05:35.575215 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575197    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 19:05:35.575302 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575282    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000"} err="failed to get container status \"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000\": rpc error: code = NotFound desc = could not find container \"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000\": container with ID starting with 5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000 not found: ID does not exist"
Jul 28 19:05:35.575340 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575302    2561 scope.go:117] "RemoveContainer" containerID="ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e"
Jul 28 19:05:35.575548 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575536    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="prometheus"
Jul 28 19:05:35.575604 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575554    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="prometheus"
Jul 28 19:05:35.575604 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575564    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="kube-rbac-proxy"
Jul 28 19:05:35.575604 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575571    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="kube-rbac-proxy"
Jul 28 19:05:35.575604 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575533    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e"} err="failed to get container status \"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e\": rpc error: code = NotFound desc = could not find container \"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e\": container with ID starting with ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e not found: ID does not exist"
Jul 28 19:05:35.575604 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575589    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="init-config-reloader"
Jul 28 19:05:35.575604 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575598    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="init-config-reloader"
Jul 28 19:05:35.575604 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575599    2561 scope.go:117] "RemoveContainer" containerID="ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8"
Jul 28 19:05:35.575855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575608    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="config-reloader"
Jul 28 19:05:35.575855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575618    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="config-reloader"
Jul 28 19:05:35.575855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575632    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="thanos-sidecar"
Jul 28 19:05:35.575855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575640    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="thanos-sidecar"
Jul 28 19:05:35.575855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575652    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="kube-rbac-proxy-thanos"
Jul 28 19:05:35.575855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575660    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="kube-rbac-proxy-thanos"
Jul 28 19:05:35.575855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575675    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="kube-rbac-proxy-web"
Jul 28 19:05:35.575855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575684    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="kube-rbac-proxy-web"
Jul 28 19:05:35.575855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575747    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="thanos-sidecar"
Jul 28 19:05:35.575855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575757    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="prometheus"
Jul 28 19:05:35.575855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575764    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="kube-rbac-proxy"
Jul 28 19:05:35.575855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575775    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="config-reloader"
Jul 28 19:05:35.575855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575783    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="kube-rbac-proxy-web"
Jul 28 19:05:35.575855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575792    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" containerName="kube-rbac-proxy-thanos"
Jul 28 19:05:35.575855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575829    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8"} err="failed to get container status \"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8\": rpc error: code = NotFound desc = could not find container \"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8\": container with ID starting with ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8 not found: ID does not exist"
Jul 28 19:05:35.575855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.575845    2561 scope.go:117] "RemoveContainer" containerID="df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e"
Jul 28 19:05:35.576381 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.576074    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e"} err="failed to get container status \"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e\": rpc error: code = NotFound desc = could not find container \"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e\": container with ID starting with df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e not found: ID does not exist"
Jul 28 19:05:35.576381 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.576088    2561 scope.go:117] "RemoveContainer" containerID="3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9"
Jul 28 19:05:35.576381 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.576295    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9"} err="failed to get container status \"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9\": rpc error: code = NotFound desc = could not find container \"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9\": container with ID starting with 3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9 not found: ID does not exist"
Jul 28 19:05:35.576381 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.576312    2561 scope.go:117] "RemoveContainer" containerID="e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad"
Jul 28 19:05:35.576565 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.576546    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad"} err="failed to get container status \"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad\": rpc error: code = NotFound desc = could not find container \"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad\": container with ID starting with e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad not found: ID does not exist"
Jul 28 19:05:35.576604 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.576565    2561 scope.go:117] "RemoveContainer" containerID="4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1"
Jul 28 19:05:35.579007 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.577134    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1"} err="failed to get container status \"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1\": rpc error: code = NotFound desc = could not find container \"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1\": container with ID starting with 4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1 not found: ID does not exist"
Jul 28 19:05:35.579007 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.577205    2561 scope.go:117] "RemoveContainer" containerID="5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000"
Jul 28 19:05:35.579178 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.579022    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000"} err="failed to get container status \"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000\": rpc error: code = NotFound desc = could not find container \"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000\": container with ID starting with 5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000 not found: ID does not exist"
Jul 28 19:05:35.579178 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.579048    2561 scope.go:117] "RemoveContainer" containerID="ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e"
Jul 28 19:05:35.580157 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.579824    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e"} err="failed to get container status \"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e\": rpc error: code = NotFound desc = could not find container \"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e\": container with ID starting with ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e not found: ID does not exist"
Jul 28 19:05:35.580157 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.579851    2561 scope.go:117] "RemoveContainer" containerID="ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8"
Jul 28 19:05:35.580323 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.580177    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8"} err="failed to get container status \"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8\": rpc error: code = NotFound desc = could not find container \"ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8\": container with ID starting with ade214c15a4c024abc611d0100406d220954c96fd9205ad6f588bb5f8f6273c8 not found: ID does not exist"
Jul 28 19:05:35.580323 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.580202    2561 scope.go:117] "RemoveContainer" containerID="df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e"
Jul 28 19:05:35.580731 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.580706    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e"} err="failed to get container status \"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e\": rpc error: code = NotFound desc = could not find container \"df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e\": container with ID starting with df3c7c0f9978edc9a485cc6e939cee51d655c0982c1e1081f54746eaf151181e not found: ID does not exist"
Jul 28 19:05:35.580731 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.580731    2561 scope.go:117] "RemoveContainer" containerID="3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9"
Jul 28 19:05:35.581132 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.581107    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9"} err="failed to get container status \"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9\": rpc error: code = NotFound desc = could not find container \"3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9\": container with ID starting with 3e46289c2b865b0c3e5e1f643eca7314e91f2444aa2ca8960b0d720baa36ede9 not found: ID does not exist"
Jul 28 19:05:35.581208 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.581133    2561 scope.go:117] "RemoveContainer" containerID="e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad"
Jul 28 19:05:35.581460 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.581437    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad"} err="failed to get container status \"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad\": rpc error: code = NotFound desc = could not find container \"e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad\": container with ID starting with e1456bd40fb93bbb8748f757f2a3474a5e147c6a112c266c899bf9688ea919ad not found: ID does not exist"
Jul 28 19:05:35.581605 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.581461    2561 scope.go:117] "RemoveContainer" containerID="4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1"
Jul 28 19:05:35.581849 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.581827    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1"} err="failed to get container status \"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1\": rpc error: code = NotFound desc = could not find container \"4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1\": container with ID starting with 4ae2e7c2a6b10335f1302b5d51d56a00a20880d59555296ab8479e1415c08be1 not found: ID does not exist"
Jul 28 19:05:35.581929 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.581849    2561 scope.go:117] "RemoveContainer" containerID="5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000"
Jul 28 19:05:35.582336 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.582314    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000"} err="failed to get container status \"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000\": rpc error: code = NotFound desc = could not find container \"5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000\": container with ID starting with 5fa368de903c12732d07fe7f54a6698520aaa14b7b611c20ce0a2c6ccd6f3000 not found: ID does not exist"
Jul 28 19:05:35.582741 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.582721    2561 scope.go:117] "RemoveContainer" containerID="ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e"
Jul 28 19:05:35.583014 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.582989    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e"} err="failed to get container status \"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e\": rpc error: code = NotFound desc = could not find container \"ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e\": container with ID starting with ea97f0685129370936b21f33ef16161e6600b2dac540a28cc6c89c6d4ce4363e not found: ID does not exist"
Jul 28 19:05:35.583088 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.583050    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.585688 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.585665    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 19:05:35.585799 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.585773    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 19:05:35.585858 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.585823    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 19:05:35.585858 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.585833    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 19:05:35.585968 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.585860    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 19:05:35.585968 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.585838    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 19:05:35.586191 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.586176    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 19:05:35.586268 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.586219    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 19:05:35.586323 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.586274    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-42fxs\""
Jul 28 19:05:35.586639 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.586621    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 19:05:35.586639 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.586637    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dahsqirpds1fu\""
Jul 28 19:05:35.586782 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.586743    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 19:05:35.586820 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.586638    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 19:05:35.587095 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.587076    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 19:05:35.589147 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.589125    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 19:05:35.592147 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.592127    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 19:05:35.595567 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.595547    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 19:05:35.679154 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679113    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679154 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679155    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679378 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679173    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679378 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679223    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679378 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679251    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679378 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679288    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f68efc2c-2a16-4f1e-9e8a-27890cff3931-config-out\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679378 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679309    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679378 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679328    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-config\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679378 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679358    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f68efc2c-2a16-4f1e-9e8a-27890cff3931-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679378 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679375    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-web-config\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679699 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679405    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679699 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679424    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679699 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679462    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679699 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679504    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679699 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679518    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679699 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679535    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679699 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679549    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679699 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679564    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf9sr\" (UniqueName: \"kubernetes.io/projected/f68efc2c-2a16-4f1e-9e8a-27890cff3931-kube-api-access-gf9sr\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679699 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679611    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.679699 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.679629    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f68efc2c-2a16-4f1e-9e8a-27890cff3931-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.780523 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780414    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f68efc2c-2a16-4f1e-9e8a-27890cff3931-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.780523 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780471    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.780523 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780520    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.780792 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780544    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.780792 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780565    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.780792 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780580    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.780792 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780604    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f68efc2c-2a16-4f1e-9e8a-27890cff3931-config-out\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.780792 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780636    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.780792 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780671    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-config\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.780792 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780699    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f68efc2c-2a16-4f1e-9e8a-27890cff3931-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.780792 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780724    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-web-config\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.780792 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780768    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.780792 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780794    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.781283 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780838    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.781283 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780880    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.781283 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780908    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.781283 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780936    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.781283 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780964    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.781283 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780989    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf9sr\" (UniqueName: \"kubernetes.io/projected/f68efc2c-2a16-4f1e-9e8a-27890cff3931-kube-api-access-gf9sr\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.781283 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.780983    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f68efc2c-2a16-4f1e-9e8a-27890cff3931-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.781283 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.781057    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.781283 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.781267    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.782086 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.781305    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.782905 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.782289    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.782905 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.782351    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.783671 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.783619    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.783816 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.783793    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.784263 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.784234    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-web-config\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.784614 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.784590    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.784709 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.784677    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.784826 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.784805    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f68efc2c-2a16-4f1e-9e8a-27890cff3931-config-out\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.785166 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.785141    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f68efc2c-2a16-4f1e-9e8a-27890cff3931-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.786042 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.785995    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-config\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.786159 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.786135    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.786227 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.786173    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.786331 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.786304    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f68efc2c-2a16-4f1e-9e8a-27890cff3931-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.786673 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.786658    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.787019 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.787001    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.787065 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.787016    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f68efc2c-2a16-4f1e-9e8a-27890cff3931-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.790249 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.790226    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf9sr\" (UniqueName: \"kubernetes.io/projected/f68efc2c-2a16-4f1e-9e8a-27890cff3931-kube-api-access-gf9sr\") pod \"prometheus-k8s-0\" (UID: \"f68efc2c-2a16-4f1e-9e8a-27890cff3931\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:35.894137 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:35.894086    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:05:36.034095 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:36.034067    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 19:05:36.035808 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:05:36.035785    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf68efc2c_2a16_4f1e_9e8a_27890cff3931.slice/crio-3fb6734edbff443ff53ed2b9431d7a9cdd4ddd94e7517c3b16b3bdec46a08ea0 WatchSource:0}: Error finding container 3fb6734edbff443ff53ed2b9431d7a9cdd4ddd94e7517c3b16b3bdec46a08ea0: Status 404 returned error can't find the container with id 3fb6734edbff443ff53ed2b9431d7a9cdd4ddd94e7517c3b16b3bdec46a08ea0
Jul 28 19:05:36.520231 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:36.520194    2561 generic.go:356] "Generic (PLEG): container finished" podID="f68efc2c-2a16-4f1e-9e8a-27890cff3931" containerID="26295842f18fca5c13b5fef5c52023cecd8c883b5d83e14989fcadfa8a2fddef" exitCode=0
Jul 28 19:05:36.520728 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:36.520243    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f68efc2c-2a16-4f1e-9e8a-27890cff3931","Type":"ContainerDied","Data":"26295842f18fca5c13b5fef5c52023cecd8c883b5d83e14989fcadfa8a2fddef"}
Jul 28 19:05:36.520728 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:36.520294    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f68efc2c-2a16-4f1e-9e8a-27890cff3931","Type":"ContainerStarted","Data":"3fb6734edbff443ff53ed2b9431d7a9cdd4ddd94e7517c3b16b3bdec46a08ea0"}
Jul 28 19:05:36.604766 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:36.604737    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc645e8b-667e-4cc4-9ea4-9c7201dfccc6" path="/var/lib/kubelet/pods/dc645e8b-667e-4cc4-9ea4-9c7201dfccc6/volumes"
Jul 28 19:05:37.527834 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:37.527798    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f68efc2c-2a16-4f1e-9e8a-27890cff3931","Type":"ContainerStarted","Data":"c6a5ddf4b055f424616a02386af01c651b4c82a280b6f955c70e116a6c6451a7"}
Jul 28 19:05:37.528253 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:37.527872    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f68efc2c-2a16-4f1e-9e8a-27890cff3931","Type":"ContainerStarted","Data":"62767e836ed68521b68ffa97c8001657862cd593b82f07ca64e5390daff4b114"}
Jul 28 19:05:37.528253 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:37.527899    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f68efc2c-2a16-4f1e-9e8a-27890cff3931","Type":"ContainerStarted","Data":"5ec68ed1abd04379ac01962c2b22d7f24879c78f2148851eff61c8fb031410c9"}
Jul 28 19:05:37.528253 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:37.527926    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f68efc2c-2a16-4f1e-9e8a-27890cff3931","Type":"ContainerStarted","Data":"eb1b8d752bb51c8098f65e1cbe8a45d011484b6586d514131e1eb505c3391831"}
Jul 28 19:05:37.528253 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:37.527949    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f68efc2c-2a16-4f1e-9e8a-27890cff3931","Type":"ContainerStarted","Data":"3eb9e7f37a2dab1e0fc7f4c140a68e8e210fd971c34eac890932f13949fca979"}
Jul 28 19:05:37.528253 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:37.527972    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f68efc2c-2a16-4f1e-9e8a-27890cff3931","Type":"ContainerStarted","Data":"24d1b8193a76d679745a47705beb7000152ce0fd972f69760d2a2aea2883c6c1"}
Jul 28 19:05:37.560496 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:37.560430    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.540427426 podStartE2EDuration="2.560407397s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.979971ms" isStatefulPod=false podCreationTimestamp="2026-07-28 19:05:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:05:37.558961373 +0000 UTC m=+185.505446265" watchObservedRunningTime="2026-07-28 19:05:37.560407397 +0000 UTC m=+185.506892270"
Jul 28 19:05:40.895087 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:05:40.895051    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:06:35.894573 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:06:35.894532    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:06:35.911113 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:06:35.911084    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:06:36.781311 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:06:36.781271    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 19:07:15.854258 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:15.854221    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"]
Jul 28 19:07:15.857941 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:15.857922    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:15.860768 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:15.860746    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 19:07:15.860935 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:15.860800    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 19:07:15.860935 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:15.860798    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 19:07:15.860935 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:15.860852    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 19:07:15.861085 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:15.860947    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2sd7z\""
Jul 28 19:07:15.871710 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:15.871687    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"]
Jul 28 19:07:15.958075 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:15.958042    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edcbf2e3-b1a1-4c87-b93f-b0c048f04797-tmp\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr\" (UID: \"edcbf2e3-b1a1-4c87-b93f-b0c048f04797\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:15.958259 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:15.958104    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/edcbf2e3-b1a1-4c87-b93f-b0c048f04797-manifests\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr\" (UID: \"edcbf2e3-b1a1-4c87-b93f-b0c048f04797\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:15.958259 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:15.958119    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/edcbf2e3-b1a1-4c87-b93f-b0c048f04797-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr\" (UID: \"edcbf2e3-b1a1-4c87-b93f-b0c048f04797\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:15.958259 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:15.958173    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/edcbf2e3-b1a1-4c87-b93f-b0c048f04797-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr\" (UID: \"edcbf2e3-b1a1-4c87-b93f-b0c048f04797\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:15.958259 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:15.958207    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcdc8\" (UniqueName: \"kubernetes.io/projected/edcbf2e3-b1a1-4c87-b93f-b0c048f04797-kube-api-access-qcdc8\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr\" (UID: \"edcbf2e3-b1a1-4c87-b93f-b0c048f04797\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:16.059604 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:16.059561    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edcbf2e3-b1a1-4c87-b93f-b0c048f04797-tmp\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr\" (UID: \"edcbf2e3-b1a1-4c87-b93f-b0c048f04797\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:16.059800 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:16.059666    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/edcbf2e3-b1a1-4c87-b93f-b0c048f04797-manifests\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr\" (UID: \"edcbf2e3-b1a1-4c87-b93f-b0c048f04797\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:16.059800 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:16.059695    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/edcbf2e3-b1a1-4c87-b93f-b0c048f04797-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr\" (UID: \"edcbf2e3-b1a1-4c87-b93f-b0c048f04797\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:16.059800 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:16.059722    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/edcbf2e3-b1a1-4c87-b93f-b0c048f04797-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr\" (UID: \"edcbf2e3-b1a1-4c87-b93f-b0c048f04797\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:16.059800 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:16.059745    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcdc8\" (UniqueName: \"kubernetes.io/projected/edcbf2e3-b1a1-4c87-b93f-b0c048f04797-kube-api-access-qcdc8\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr\" (UID: \"edcbf2e3-b1a1-4c87-b93f-b0c048f04797\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:16.060102 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:16.060068    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/edcbf2e3-b1a1-4c87-b93f-b0c048f04797-manifests\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr\" (UID: \"edcbf2e3-b1a1-4c87-b93f-b0c048f04797\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:16.061922 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:16.061891    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edcbf2e3-b1a1-4c87-b93f-b0c048f04797-tmp\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr\" (UID: \"edcbf2e3-b1a1-4c87-b93f-b0c048f04797\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:16.062174 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:16.062153    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/edcbf2e3-b1a1-4c87-b93f-b0c048f04797-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr\" (UID: \"edcbf2e3-b1a1-4c87-b93f-b0c048f04797\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:16.062243 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:16.062177    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/edcbf2e3-b1a1-4c87-b93f-b0c048f04797-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr\" (UID: \"edcbf2e3-b1a1-4c87-b93f-b0c048f04797\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:16.068395 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:16.068373    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcdc8\" (UniqueName: \"kubernetes.io/projected/edcbf2e3-b1a1-4c87-b93f-b0c048f04797-kube-api-access-qcdc8\") pod \"opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr\" (UID: \"edcbf2e3-b1a1-4c87-b93f-b0c048f04797\") " pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:16.168524 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:16.168414    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:16.308717 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:16.308685    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"]
Jul 28 19:07:16.311218 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:07:16.311189    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedcbf2e3_b1a1_4c87_b93f_b0c048f04797.slice/crio-bdea9b87ccd74b0678aee9078b325835699aaf2387b9baef8cf84de838c3bd0c WatchSource:0}: Error finding container bdea9b87ccd74b0678aee9078b325835699aaf2387b9baef8cf84de838c3bd0c: Status 404 returned error can't find the container with id bdea9b87ccd74b0678aee9078b325835699aaf2387b9baef8cf84de838c3bd0c
Jul 28 19:07:16.920091 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:16.920056    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr" event={"ID":"edcbf2e3-b1a1-4c87-b93f-b0c048f04797","Type":"ContainerStarted","Data":"bdea9b87ccd74b0678aee9078b325835699aaf2387b9baef8cf84de838c3bd0c"}
Jul 28 19:07:19.936964 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:19.936931    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr" event={"ID":"edcbf2e3-b1a1-4c87-b93f-b0c048f04797","Type":"ContainerStarted","Data":"4cc3e2b2d61b04305eab418ca4bdffbf54db32b37ed98bde94b3939405c30bc0"}
Jul 28 19:07:20.942588 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:20.942553    2561 generic.go:356] "Generic (PLEG): container finished" podID="edcbf2e3-b1a1-4c87-b93f-b0c048f04797" containerID="4cc3e2b2d61b04305eab418ca4bdffbf54db32b37ed98bde94b3939405c30bc0" exitCode=0
Jul 28 19:07:20.943042 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:20.942627    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr" event={"ID":"edcbf2e3-b1a1-4c87-b93f-b0c048f04797","Type":"ContainerDied","Data":"4cc3e2b2d61b04305eab418ca4bdffbf54db32b37ed98bde94b3939405c30bc0"}
Jul 28 19:07:21.949667 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:21.949628    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr" event={"ID":"edcbf2e3-b1a1-4c87-b93f-b0c048f04797","Type":"ContainerStarted","Data":"ae13c2232ad15ad71f31c0c468f2d23857db0d0bdb6b8d39dbec72b0e2fe5237"}
Jul 28 19:07:21.950098 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:21.949732    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:21.974913 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:21.974856    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr" podStartSLOduration=2.376327757 podStartE2EDuration="6.974834845s" totalImagesPullingTime="3.504724549s" totalInitContainerRuntime="1.093782539s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:07:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:07:21.972843908 +0000 UTC m=+289.919328776" watchObservedRunningTime="2026-07-28 19:07:21.974834845 +0000 UTC m=+289.921319713"
Jul 28 19:07:32.388855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:32.388821    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:07:32.396057 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:32.396025    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:07:32.399843 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:32.399823    2561 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 19:07:32.400216 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:32.400198    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:07:32.407632 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:32.407578    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:07:32.956535 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:32.956505    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr"
Jul 28 19:07:33.558167 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.558133    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"]
Jul 28 19:07:33.560299 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.560284    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"
Jul 28 19:07:33.563551 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.563531    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 19:07:33.563749 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.563737    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 19:07:33.564697 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.564678    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 19:07:33.564815 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.564800    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:07:33.564875 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.564856    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kk6wv\""
Jul 28 19:07:33.565734 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.565716    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 19:07:33.575160 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.575138    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"]
Jul 28 19:07:33.605339 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.605305    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2gvg\" (UniqueName: \"kubernetes.io/projected/e07ee69b-85e7-485f-8072-ea7010acea97-kube-api-access-c2gvg\") pod \"lws-controller-manager-7fbf845658-x9wsg\" (UID: \"e07ee69b-85e7-485f-8072-ea7010acea97\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"
Jul 28 19:07:33.605519 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.605354    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e07ee69b-85e7-485f-8072-ea7010acea97-cert\") pod \"lws-controller-manager-7fbf845658-x9wsg\" (UID: \"e07ee69b-85e7-485f-8072-ea7010acea97\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"
Jul 28 19:07:33.605519 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.605375    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e07ee69b-85e7-485f-8072-ea7010acea97-manager-config\") pod \"lws-controller-manager-7fbf845658-x9wsg\" (UID: \"e07ee69b-85e7-485f-8072-ea7010acea97\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"
Jul 28 19:07:33.605519 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.605407    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e07ee69b-85e7-485f-8072-ea7010acea97-metrics-cert\") pod \"lws-controller-manager-7fbf845658-x9wsg\" (UID: \"e07ee69b-85e7-485f-8072-ea7010acea97\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"
Jul 28 19:07:33.706805 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.706770    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e07ee69b-85e7-485f-8072-ea7010acea97-cert\") pod \"lws-controller-manager-7fbf845658-x9wsg\" (UID: \"e07ee69b-85e7-485f-8072-ea7010acea97\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"
Jul 28 19:07:33.706805 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.706809    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e07ee69b-85e7-485f-8072-ea7010acea97-manager-config\") pod \"lws-controller-manager-7fbf845658-x9wsg\" (UID: \"e07ee69b-85e7-485f-8072-ea7010acea97\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"
Jul 28 19:07:33.707053 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.706831    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e07ee69b-85e7-485f-8072-ea7010acea97-metrics-cert\") pod \"lws-controller-manager-7fbf845658-x9wsg\" (UID: \"e07ee69b-85e7-485f-8072-ea7010acea97\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"
Jul 28 19:07:33.707053 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.706964    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2gvg\" (UniqueName: \"kubernetes.io/projected/e07ee69b-85e7-485f-8072-ea7010acea97-kube-api-access-c2gvg\") pod \"lws-controller-manager-7fbf845658-x9wsg\" (UID: \"e07ee69b-85e7-485f-8072-ea7010acea97\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"
Jul 28 19:07:33.707624 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.707599    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e07ee69b-85e7-485f-8072-ea7010acea97-manager-config\") pod \"lws-controller-manager-7fbf845658-x9wsg\" (UID: \"e07ee69b-85e7-485f-8072-ea7010acea97\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"
Jul 28 19:07:33.709303 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.709283    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e07ee69b-85e7-485f-8072-ea7010acea97-metrics-cert\") pod \"lws-controller-manager-7fbf845658-x9wsg\" (UID: \"e07ee69b-85e7-485f-8072-ea7010acea97\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"
Jul 28 19:07:33.709410 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.709393    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e07ee69b-85e7-485f-8072-ea7010acea97-cert\") pod \"lws-controller-manager-7fbf845658-x9wsg\" (UID: \"e07ee69b-85e7-485f-8072-ea7010acea97\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"
Jul 28 19:07:33.714979 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.714961    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2gvg\" (UniqueName: \"kubernetes.io/projected/e07ee69b-85e7-485f-8072-ea7010acea97-kube-api-access-c2gvg\") pod \"lws-controller-manager-7fbf845658-x9wsg\" (UID: \"e07ee69b-85e7-485f-8072-ea7010acea97\") " pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"
Jul 28 19:07:33.869333 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:33.869246    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"
Jul 28 19:07:34.000064 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:34.000034    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"]
Jul 28 19:07:34.003018 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:07:34.002994    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode07ee69b_85e7_485f_8072_ea7010acea97.slice/crio-69c55e89d2a9c8eec422f669329bc019e2f6781120ada0d104d93817833181bc WatchSource:0}: Error finding container 69c55e89d2a9c8eec422f669329bc019e2f6781120ada0d104d93817833181bc: Status 404 returned error can't find the container with id 69c55e89d2a9c8eec422f669329bc019e2f6781120ada0d104d93817833181bc
Jul 28 19:07:34.005390 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:34.005374    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:07:35.009354 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:35.009312    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg" event={"ID":"e07ee69b-85e7-485f-8072-ea7010acea97","Type":"ContainerStarted","Data":"69c55e89d2a9c8eec422f669329bc019e2f6781120ada0d104d93817833181bc"}
Jul 28 19:07:35.956560 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:35.956515    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b4694559d-thlwz"]
Jul 28 19:07:35.962754 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:35.962721    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b4694559d-thlwz"
Jul 28 19:07:35.966669 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:35.966435    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 19:07:35.967183 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:35.967156    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 19:07:35.967345 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:35.967204    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gxk5p\""
Jul 28 19:07:36.021321 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:36.021286    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b4694559d-thlwz"]
Jul 28 19:07:36.028093 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:36.028064    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b22490-f769-431e-b092-90624a1b08a9-tls-certs\") pod \"kube-auth-proxy-5b4694559d-thlwz\" (UID: \"33b22490-f769-431e-b092-90624a1b08a9\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-thlwz"
Jul 28 19:07:36.028262 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:36.028104    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff9zc\" (UniqueName: \"kubernetes.io/projected/33b22490-f769-431e-b092-90624a1b08a9-kube-api-access-ff9zc\") pod \"kube-auth-proxy-5b4694559d-thlwz\" (UID: \"33b22490-f769-431e-b092-90624a1b08a9\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-thlwz"
Jul 28 19:07:36.028262 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:36.028142    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33b22490-f769-431e-b092-90624a1b08a9-tmp\") pod \"kube-auth-proxy-5b4694559d-thlwz\" (UID: \"33b22490-f769-431e-b092-90624a1b08a9\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-thlwz"
Jul 28 19:07:36.129737 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:36.129698    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b22490-f769-431e-b092-90624a1b08a9-tls-certs\") pod \"kube-auth-proxy-5b4694559d-thlwz\" (UID: \"33b22490-f769-431e-b092-90624a1b08a9\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-thlwz"
Jul 28 19:07:36.129737 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:36.129747    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff9zc\" (UniqueName: \"kubernetes.io/projected/33b22490-f769-431e-b092-90624a1b08a9-kube-api-access-ff9zc\") pod \"kube-auth-proxy-5b4694559d-thlwz\" (UID: \"33b22490-f769-431e-b092-90624a1b08a9\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-thlwz"
Jul 28 19:07:36.130000 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:36.129784    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33b22490-f769-431e-b092-90624a1b08a9-tmp\") pod \"kube-auth-proxy-5b4694559d-thlwz\" (UID: \"33b22490-f769-431e-b092-90624a1b08a9\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-thlwz"
Jul 28 19:07:36.130000 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:07:36.129938    2561 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 28 19:07:36.130101 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:07:36.130024    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33b22490-f769-431e-b092-90624a1b08a9-tls-certs podName:33b22490-f769-431e-b092-90624a1b08a9 nodeName:}" failed. No retries permitted until 2026-07-28 19:07:36.630001252 +0000 UTC m=+304.576486120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/33b22490-f769-431e-b092-90624a1b08a9-tls-certs") pod "kube-auth-proxy-5b4694559d-thlwz" (UID: "33b22490-f769-431e-b092-90624a1b08a9") : secret "kube-auth-proxy-tls" not found
Jul 28 19:07:36.132925 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:36.132867    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33b22490-f769-431e-b092-90624a1b08a9-tmp\") pod \"kube-auth-proxy-5b4694559d-thlwz\" (UID: \"33b22490-f769-431e-b092-90624a1b08a9\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-thlwz"
Jul 28 19:07:36.142589 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:36.142559    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff9zc\" (UniqueName: \"kubernetes.io/projected/33b22490-f769-431e-b092-90624a1b08a9-kube-api-access-ff9zc\") pod \"kube-auth-proxy-5b4694559d-thlwz\" (UID: \"33b22490-f769-431e-b092-90624a1b08a9\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-thlwz"
Jul 28 19:07:36.635240 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:36.635204    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b22490-f769-431e-b092-90624a1b08a9-tls-certs\") pod \"kube-auth-proxy-5b4694559d-thlwz\" (UID: \"33b22490-f769-431e-b092-90624a1b08a9\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-thlwz"
Jul 28 19:07:36.644693 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:36.644099    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b22490-f769-431e-b092-90624a1b08a9-tls-certs\") pod \"kube-auth-proxy-5b4694559d-thlwz\" (UID: \"33b22490-f769-431e-b092-90624a1b08a9\") " pod="openshift-ingress/kube-auth-proxy-5b4694559d-thlwz"
Jul 28 19:07:36.876088 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:36.876006    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b4694559d-thlwz"
Jul 28 19:07:37.011625 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:37.011590    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b4694559d-thlwz"]
Jul 28 19:07:37.014402 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:07:37.014364    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33b22490_f769_431e_b092_90624a1b08a9.slice/crio-6228940f226c9084503553f1b8fa2233ca695c53b848b3652d8037a682f61390 WatchSource:0}: Error finding container 6228940f226c9084503553f1b8fa2233ca695c53b848b3652d8037a682f61390: Status 404 returned error can't find the container with id 6228940f226c9084503553f1b8fa2233ca695c53b848b3652d8037a682f61390
Jul 28 19:07:37.020117 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:37.020091    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg" event={"ID":"e07ee69b-85e7-485f-8072-ea7010acea97","Type":"ContainerStarted","Data":"af400b676513fa8361d79cc608b3d27a8b254724b852c328f113353cbcd54995"}
Jul 28 19:07:37.020277 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:37.020265    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"
Jul 28 19:07:37.040588 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:37.040531    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg" podStartSLOduration=1.427854572 podStartE2EDuration="4.040514781s" totalImagesPullingTime="2.612660209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:07:37.039186935 +0000 UTC m=+304.985671826" watchObservedRunningTime="2026-07-28 19:07:37.040514781 +0000 UTC m=+304.986999641"
Jul 28 19:07:38.026559 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:38.026513    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b4694559d-thlwz" event={"ID":"33b22490-f769-431e-b092-90624a1b08a9","Type":"ContainerStarted","Data":"6228940f226c9084503553f1b8fa2233ca695c53b848b3652d8037a682f61390"}
Jul 28 19:07:40.037686 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:40.037647    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b4694559d-thlwz" event={"ID":"33b22490-f769-431e-b092-90624a1b08a9","Type":"ContainerStarted","Data":"e0a4c9e77df364af66af4e53b475112c4fba70e50b31136cbb1b02b3a65b8ff0"}
Jul 28 19:07:40.055238 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:40.055180    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b4694559d-thlwz" podStartSLOduration=2.38819815 podStartE2EDuration="5.05515945s" totalImagesPullingTime="2.6669613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:07:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:07:40.053223629 +0000 UTC m=+307.999708519" watchObservedRunningTime="2026-07-28 19:07:40.05515945 +0000 UTC m=+308.001644319"
Jul 28 19:07:48.028555 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:07:48.028520    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7fbf845658-x9wsg"
Jul 28 19:09:40.307798 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:40.307766    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"]
Jul 28 19:09:40.310068 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:40.310046    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"
Jul 28 19:09:40.313853 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:40.313833    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 19:09:40.314078 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:40.314065    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 19:09:40.314452 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:40.314437    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6s92b\""
Jul 28 19:09:40.325495 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:40.325461    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"]
Jul 28 19:09:40.358088 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:40.358061    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s55h5\" (UniqueName: \"kubernetes.io/projected/605020fe-c660-4804-869e-3e1c23d5cf88-kube-api-access-s55h5\") pod \"kuadrant-operator-controller-manager-86878c769f-g88px\" (UID: \"605020fe-c660-4804-869e-3e1c23d5cf88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"
Jul 28 19:09:40.358173 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:40.358105    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/605020fe-c660-4804-869e-3e1c23d5cf88-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g88px\" (UID: \"605020fe-c660-4804-869e-3e1c23d5cf88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"
Jul 28 19:09:40.458524 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:40.458479    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s55h5\" (UniqueName: \"kubernetes.io/projected/605020fe-c660-4804-869e-3e1c23d5cf88-kube-api-access-s55h5\") pod \"kuadrant-operator-controller-manager-86878c769f-g88px\" (UID: \"605020fe-c660-4804-869e-3e1c23d5cf88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"
Jul 28 19:09:40.458620 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:40.458549    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/605020fe-c660-4804-869e-3e1c23d5cf88-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g88px\" (UID: \"605020fe-c660-4804-869e-3e1c23d5cf88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"
Jul 28 19:09:40.458857 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:40.458842    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/605020fe-c660-4804-869e-3e1c23d5cf88-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g88px\" (UID: \"605020fe-c660-4804-869e-3e1c23d5cf88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"
Jul 28 19:09:40.473121 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:40.473091    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s55h5\" (UniqueName: \"kubernetes.io/projected/605020fe-c660-4804-869e-3e1c23d5cf88-kube-api-access-s55h5\") pod \"kuadrant-operator-controller-manager-86878c769f-g88px\" (UID: \"605020fe-c660-4804-869e-3e1c23d5cf88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"
Jul 28 19:09:40.620729 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:40.620672    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"
Jul 28 19:09:40.753334 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:40.753303    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"]
Jul 28 19:09:40.755871 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:09:40.755846    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod605020fe_c660_4804_869e_3e1c23d5cf88.slice/crio-34e4c4c5d4381ea95f568131516f9c180c24ad61412327cf37909f0d23457bf8 WatchSource:0}: Error finding container 34e4c4c5d4381ea95f568131516f9c180c24ad61412327cf37909f0d23457bf8: Status 404 returned error can't find the container with id 34e4c4c5d4381ea95f568131516f9c180c24ad61412327cf37909f0d23457bf8
Jul 28 19:09:41.566832 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:41.566798    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px" event={"ID":"605020fe-c660-4804-869e-3e1c23d5cf88","Type":"ContainerStarted","Data":"34e4c4c5d4381ea95f568131516f9c180c24ad61412327cf37909f0d23457bf8"}
Jul 28 19:09:45.592268 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:45.592235    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px" event={"ID":"605020fe-c660-4804-869e-3e1c23d5cf88","Type":"ContainerStarted","Data":"0bebb009ae154c65e461359be90ccdbca575e263fff18cb4ef46c899bcb7f0fc"}
Jul 28 19:09:45.592709 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:45.592315    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"
Jul 28 19:09:45.614388 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:45.614334    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px" podStartSLOduration=1.230533189 podStartE2EDuration="5.614317913s" totalImagesPullingTime="4.383784724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:09:45.612138546 +0000 UTC m=+433.558623413" watchObservedRunningTime="2026-07-28 19:09:45.614317913 +0000 UTC m=+433.560802778"
Jul 28 19:09:56.603104 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:09:56.603036    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"
Jul 28 19:10:16.611766 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:16.611737    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-2snhz"]
Jul 28 19:10:16.622357 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:16.622327    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-2snhz"
Jul 28 19:10:16.625429 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:16.625408    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-r8kdh\""
Jul 28 19:10:16.627981 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:16.627962    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2snhz"]
Jul 28 19:10:16.751292 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:16.751266    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trt5l\" (UniqueName: \"kubernetes.io/projected/98519a5c-d297-40b6-b489-f1af85821a21-kube-api-access-trt5l\") pod \"authorino-75d556bd77-2snhz\" (UID: \"98519a5c-d297-40b6-b489-f1af85821a21\") " pod="kuadrant-system/authorino-75d556bd77-2snhz"
Jul 28 19:10:16.851728 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:16.851704    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trt5l\" (UniqueName: \"kubernetes.io/projected/98519a5c-d297-40b6-b489-f1af85821a21-kube-api-access-trt5l\") pod \"authorino-75d556bd77-2snhz\" (UID: \"98519a5c-d297-40b6-b489-f1af85821a21\") " pod="kuadrant-system/authorino-75d556bd77-2snhz"
Jul 28 19:10:16.859576 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:16.859553    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trt5l\" (UniqueName: \"kubernetes.io/projected/98519a5c-d297-40b6-b489-f1af85821a21-kube-api-access-trt5l\") pod \"authorino-75d556bd77-2snhz\" (UID: \"98519a5c-d297-40b6-b489-f1af85821a21\") " pod="kuadrant-system/authorino-75d556bd77-2snhz"
Jul 28 19:10:16.932836 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:16.932786    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-2snhz"
Jul 28 19:10:17.055595 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:17.055569    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2snhz"]
Jul 28 19:10:17.058145 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:10:17.058119    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98519a5c_d297_40b6_b489_f1af85821a21.slice/crio-a79891069f51ac054519fada17ceae660fca68c11e101f06108544dee1a774ec WatchSource:0}: Error finding container a79891069f51ac054519fada17ceae660fca68c11e101f06108544dee1a774ec: Status 404 returned error can't find the container with id a79891069f51ac054519fada17ceae660fca68c11e101f06108544dee1a774ec
Jul 28 19:10:17.736391 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:17.736357    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-2snhz" event={"ID":"98519a5c-d297-40b6-b489-f1af85821a21","Type":"ContainerStarted","Data":"a79891069f51ac054519fada17ceae660fca68c11e101f06108544dee1a774ec"}
Jul 28 19:10:20.753338 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:20.753304    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-2snhz" event={"ID":"98519a5c-d297-40b6-b489-f1af85821a21","Type":"ContainerStarted","Data":"91c68ecc4d88eadd7677b90ce18cd9adaa299949d1ea2abb973ae6aadb82ea00"}
Jul 28 19:10:20.775579 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:20.775534    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-2snhz" podStartSLOduration=1.783070769 podStartE2EDuration="4.775520038s" totalImagesPullingTime="2.992449269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:10:20.774811419 +0000 UTC m=+468.721296286" watchObservedRunningTime="2026-07-28 19:10:20.775520038 +0000 UTC m=+468.722004904"
Jul 28 19:10:44.409782 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.409734    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-bstf5"]
Jul 28 19:10:44.413302 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.413281    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-bstf5"
Jul 28 19:10:44.415872 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.415843    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 28 19:10:44.419847 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.419825    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-bstf5"]
Jul 28 19:10:44.438519 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.438469    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-bstf5"]
Jul 28 19:10:44.438787 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:10:44.438763    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-m62kg tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-m62kg tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-bstf5" podUID="52086628-99c7-4cf2-aefe-f451e1d44dbf"
Jul 28 19:10:44.460627 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.460596    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52086628-99c7-4cf2-aefe-f451e1d44dbf-tls-cert\") pod \"authorino-76877994-bstf5\" (UID: \"52086628-99c7-4cf2-aefe-f451e1d44dbf\") " pod="kuadrant-system/authorino-76877994-bstf5"
Jul 28 19:10:44.460778 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.460638    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m62kg\" (UniqueName: \"kubernetes.io/projected/52086628-99c7-4cf2-aefe-f451e1d44dbf-kube-api-access-m62kg\") pod \"authorino-76877994-bstf5\" (UID: \"52086628-99c7-4cf2-aefe-f451e1d44dbf\") " pod="kuadrant-system/authorino-76877994-bstf5"
Jul 28 19:10:44.561594 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.561558    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52086628-99c7-4cf2-aefe-f451e1d44dbf-tls-cert\") pod \"authorino-76877994-bstf5\" (UID: \"52086628-99c7-4cf2-aefe-f451e1d44dbf\") " pod="kuadrant-system/authorino-76877994-bstf5"
Jul 28 19:10:44.561594 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.561601    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m62kg\" (UniqueName: \"kubernetes.io/projected/52086628-99c7-4cf2-aefe-f451e1d44dbf-kube-api-access-m62kg\") pod \"authorino-76877994-bstf5\" (UID: \"52086628-99c7-4cf2-aefe-f451e1d44dbf\") " pod="kuadrant-system/authorino-76877994-bstf5"
Jul 28 19:10:44.564061 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.564041    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52086628-99c7-4cf2-aefe-f451e1d44dbf-tls-cert\") pod \"authorino-76877994-bstf5\" (UID: \"52086628-99c7-4cf2-aefe-f451e1d44dbf\") " pod="kuadrant-system/authorino-76877994-bstf5"
Jul 28 19:10:44.569542 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.569515    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m62kg\" (UniqueName: \"kubernetes.io/projected/52086628-99c7-4cf2-aefe-f451e1d44dbf-kube-api-access-m62kg\") pod \"authorino-76877994-bstf5\" (UID: \"52086628-99c7-4cf2-aefe-f451e1d44dbf\") " pod="kuadrant-system/authorino-76877994-bstf5"
Jul 28 19:10:44.855798 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.855768    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-bstf5"
Jul 28 19:10:44.862744 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.862723    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-bstf5"
Jul 28 19:10:44.964852 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.964819    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m62kg\" (UniqueName: \"kubernetes.io/projected/52086628-99c7-4cf2-aefe-f451e1d44dbf-kube-api-access-m62kg\") pod \"52086628-99c7-4cf2-aefe-f451e1d44dbf\" (UID: \"52086628-99c7-4cf2-aefe-f451e1d44dbf\") "
Jul 28 19:10:44.965054 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.964859    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52086628-99c7-4cf2-aefe-f451e1d44dbf-tls-cert\") pod \"52086628-99c7-4cf2-aefe-f451e1d44dbf\" (UID: \"52086628-99c7-4cf2-aefe-f451e1d44dbf\") "
Jul 28 19:10:44.966843 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.966811    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52086628-99c7-4cf2-aefe-f451e1d44dbf-kube-api-access-m62kg" (OuterVolumeSpecName: "kube-api-access-m62kg") pod "52086628-99c7-4cf2-aefe-f451e1d44dbf" (UID: "52086628-99c7-4cf2-aefe-f451e1d44dbf"). InnerVolumeSpecName "kube-api-access-m62kg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:10:44.966988 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:44.966963    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52086628-99c7-4cf2-aefe-f451e1d44dbf-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "52086628-99c7-4cf2-aefe-f451e1d44dbf" (UID: "52086628-99c7-4cf2-aefe-f451e1d44dbf"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:10:45.065613 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:45.065580    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m62kg\" (UniqueName: \"kubernetes.io/projected/52086628-99c7-4cf2-aefe-f451e1d44dbf-kube-api-access-m62kg\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:10:45.065613 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:45.065610    2561 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52086628-99c7-4cf2-aefe-f451e1d44dbf-tls-cert\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:10:45.743176 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:45.743141    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2snhz"]
Jul 28 19:10:45.743596 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:45.743309    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-2snhz" podUID="98519a5c-d297-40b6-b489-f1af85821a21" containerName="authorino" containerID="cri-o://91c68ecc4d88eadd7677b90ce18cd9adaa299949d1ea2abb973ae6aadb82ea00" gracePeriod=30
Jul 28 19:10:45.861737 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:45.861703    2561 generic.go:356] "Generic (PLEG): container finished" podID="98519a5c-d297-40b6-b489-f1af85821a21" containerID="91c68ecc4d88eadd7677b90ce18cd9adaa299949d1ea2abb973ae6aadb82ea00" exitCode=0
Jul 28 19:10:45.861851 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:45.861770    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-2snhz" event={"ID":"98519a5c-d297-40b6-b489-f1af85821a21","Type":"ContainerDied","Data":"91c68ecc4d88eadd7677b90ce18cd9adaa299949d1ea2abb973ae6aadb82ea00"}
Jul 28 19:10:45.861890 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:45.861868    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-bstf5"
Jul 28 19:10:45.895972 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:45.895941    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-bstf5"]
Jul 28 19:10:45.899219 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:45.899188    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-bstf5"]
Jul 28 19:10:45.994910 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:45.994853    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-2snhz"
Jul 28 19:10:46.075468 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:46.075434    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trt5l\" (UniqueName: \"kubernetes.io/projected/98519a5c-d297-40b6-b489-f1af85821a21-kube-api-access-trt5l\") pod \"98519a5c-d297-40b6-b489-f1af85821a21\" (UID: \"98519a5c-d297-40b6-b489-f1af85821a21\") "
Jul 28 19:10:46.077622 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:46.077584    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98519a5c-d297-40b6-b489-f1af85821a21-kube-api-access-trt5l" (OuterVolumeSpecName: "kube-api-access-trt5l") pod "98519a5c-d297-40b6-b489-f1af85821a21" (UID: "98519a5c-d297-40b6-b489-f1af85821a21"). InnerVolumeSpecName "kube-api-access-trt5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:10:46.176191 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:46.176154    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trt5l\" (UniqueName: \"kubernetes.io/projected/98519a5c-d297-40b6-b489-f1af85821a21-kube-api-access-trt5l\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:10:46.602862 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:46.602832    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52086628-99c7-4cf2-aefe-f451e1d44dbf" path="/var/lib/kubelet/pods/52086628-99c7-4cf2-aefe-f451e1d44dbf/volumes"
Jul 28 19:10:46.867610 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:46.867520    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-2snhz" event={"ID":"98519a5c-d297-40b6-b489-f1af85821a21","Type":"ContainerDied","Data":"a79891069f51ac054519fada17ceae660fca68c11e101f06108544dee1a774ec"}
Jul 28 19:10:46.867610 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:46.867580    2561 scope.go:117] "RemoveContainer" containerID="91c68ecc4d88eadd7677b90ce18cd9adaa299949d1ea2abb973ae6aadb82ea00"
Jul 28 19:10:46.868133 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:46.867529    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-2snhz"
Jul 28 19:10:46.884256 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:46.884234    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2snhz"]
Jul 28 19:10:46.888421 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:46.888400    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2snhz"]
Jul 28 19:10:48.603189 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:10:48.603152    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98519a5c-d297-40b6-b489-f1af85821a21" path="/var/lib/kubelet/pods/98519a5c-d297-40b6-b489-f1af85821a21/volumes"
Jul 28 19:11:26.458464 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:26.458392    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-jdvzc"]
Jul 28 19:11:26.458895 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:26.458740    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98519a5c-d297-40b6-b489-f1af85821a21" containerName="authorino"
Jul 28 19:11:26.458895 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:26.458752    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="98519a5c-d297-40b6-b489-f1af85821a21" containerName="authorino"
Jul 28 19:11:26.458895 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:26.458822    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="98519a5c-d297-40b6-b489-f1af85821a21" containerName="authorino"
Jul 28 19:11:26.461673 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:26.461657    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-jdvzc"
Jul 28 19:11:26.464176 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:26.464155    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-cw4nb\""
Jul 28 19:11:26.464305 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:26.464185    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 28 19:11:26.480780 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:26.480758    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-jdvzc"]
Jul 28 19:11:26.593476 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:26.593450    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djjmp\" (UniqueName: \"kubernetes.io/projected/bec6473a-99eb-4911-9557-3ba753ed35eb-kube-api-access-djjmp\") pod \"payload-processing-5cc6cd444b-jdvzc\" (UID: \"bec6473a-99eb-4911-9557-3ba753ed35eb\") " pod="openshift-ingress/payload-processing-5cc6cd444b-jdvzc"
Jul 28 19:11:26.593600 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:26.593546    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bec6473a-99eb-4911-9557-3ba753ed35eb-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-jdvzc\" (UID: \"bec6473a-99eb-4911-9557-3ba753ed35eb\") " pod="openshift-ingress/payload-processing-5cc6cd444b-jdvzc"
Jul 28 19:11:26.694375 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:26.694344    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bec6473a-99eb-4911-9557-3ba753ed35eb-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-jdvzc\" (UID: \"bec6473a-99eb-4911-9557-3ba753ed35eb\") " pod="openshift-ingress/payload-processing-5cc6cd444b-jdvzc"
Jul 28 19:11:26.694522 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:26.694398    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djjmp\" (UniqueName: \"kubernetes.io/projected/bec6473a-99eb-4911-9557-3ba753ed35eb-kube-api-access-djjmp\") pod \"payload-processing-5cc6cd444b-jdvzc\" (UID: \"bec6473a-99eb-4911-9557-3ba753ed35eb\") " pod="openshift-ingress/payload-processing-5cc6cd444b-jdvzc"
Jul 28 19:11:26.694993 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:26.694977    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bec6473a-99eb-4911-9557-3ba753ed35eb-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-jdvzc\" (UID: \"bec6473a-99eb-4911-9557-3ba753ed35eb\") " pod="openshift-ingress/payload-processing-5cc6cd444b-jdvzc"
Jul 28 19:11:26.710663 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:26.710609    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djjmp\" (UniqueName: \"kubernetes.io/projected/bec6473a-99eb-4911-9557-3ba753ed35eb-kube-api-access-djjmp\") pod \"payload-processing-5cc6cd444b-jdvzc\" (UID: \"bec6473a-99eb-4911-9557-3ba753ed35eb\") " pod="openshift-ingress/payload-processing-5cc6cd444b-jdvzc"
Jul 28 19:11:26.771260 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:26.771240    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-jdvzc"
Jul 28 19:11:26.902332 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:26.902290    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-jdvzc"]
Jul 28 19:11:26.905851 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:11:26.905823    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbec6473a_99eb_4911_9557_3ba753ed35eb.slice/crio-712b112a2cc4812d7169007d9cee2bea73d4f00014a49c206930013697b2a798 WatchSource:0}: Error finding container 712b112a2cc4812d7169007d9cee2bea73d4f00014a49c206930013697b2a798: Status 404 returned error can't find the container with id 712b112a2cc4812d7169007d9cee2bea73d4f00014a49c206930013697b2a798
Jul 28 19:11:27.042064 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:27.041993    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-jdvzc" event={"ID":"bec6473a-99eb-4911-9557-3ba753ed35eb","Type":"ContainerStarted","Data":"712b112a2cc4812d7169007d9cee2bea73d4f00014a49c206930013697b2a798"}
Jul 28 19:11:30.059711 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:30.059677    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-jdvzc" event={"ID":"bec6473a-99eb-4911-9557-3ba753ed35eb","Type":"ContainerStarted","Data":"763d19e3f8528f7ee76b27cffbbf83f8a1f52eec01996ef26a82a92139a359d7"}
Jul 28 19:11:30.060100 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:30.059821    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-jdvzc"
Jul 28 19:11:30.077038 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:30.076991    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-jdvzc" podStartSLOduration=1.8727157920000002 podStartE2EDuration="4.07697734s" totalImagesPullingTime="2.204261548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:11:30.075714448 +0000 UTC m=+538.022199341" watchObservedRunningTime="2026-07-28 19:11:30.07697734 +0000 UTC m=+538.023462214"
Jul 28 19:11:40.953136 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:40.953099    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"]
Jul 28 19:11:40.957999 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:40.957975    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:40.961440 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:40.961422    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 19:11:40.961554 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:40.961447    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-twk7j\""
Jul 28 19:11:40.961554 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:40.961458    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 28 19:11:40.961554 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:40.961454    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 19:11:40.968938 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:40.968916    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"]
Jul 28 19:11:41.065986 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.065958    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-jdvzc"
Jul 28 19:11:41.106597 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.106563    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/322324a7-e80f-456e-b0da-5ddc45698b55-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.106754 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.106613    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htn2r\" (UniqueName: \"kubernetes.io/projected/322324a7-e80f-456e-b0da-5ddc45698b55-kube-api-access-htn2r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.106754 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.106642    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/322324a7-e80f-456e-b0da-5ddc45698b55-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.106754 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.106717    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/322324a7-e80f-456e-b0da-5ddc45698b55-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.106881 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.106757    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/322324a7-e80f-456e-b0da-5ddc45698b55-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.106881 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.106834    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/322324a7-e80f-456e-b0da-5ddc45698b55-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.207376 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.207276    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/322324a7-e80f-456e-b0da-5ddc45698b55-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.207376 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.207335    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htn2r\" (UniqueName: \"kubernetes.io/projected/322324a7-e80f-456e-b0da-5ddc45698b55-kube-api-access-htn2r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.207376 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.207364    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/322324a7-e80f-456e-b0da-5ddc45698b55-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.207712 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.207548    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/322324a7-e80f-456e-b0da-5ddc45698b55-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.207712 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.207582    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/322324a7-e80f-456e-b0da-5ddc45698b55-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.207712 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.207643    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/322324a7-e80f-456e-b0da-5ddc45698b55-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.207906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.207819    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/322324a7-e80f-456e-b0da-5ddc45698b55-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.207906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.207898    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/322324a7-e80f-456e-b0da-5ddc45698b55-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.208067 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.207975    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/322324a7-e80f-456e-b0da-5ddc45698b55-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.209795 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.209769    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/322324a7-e80f-456e-b0da-5ddc45698b55-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.209978 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.209961    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/322324a7-e80f-456e-b0da-5ddc45698b55-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.216435 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.216410    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htn2r\" (UniqueName: \"kubernetes.io/projected/322324a7-e80f-456e-b0da-5ddc45698b55-kube-api-access-htn2r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt\" (UID: \"322324a7-e80f-456e-b0da-5ddc45698b55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.267878 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.267851    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:41.419404 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:41.419369    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"]
Jul 28 19:11:41.422455 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:11:41.422424    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod322324a7_e80f_456e_b0da_5ddc45698b55.slice/crio-71b6aa07b2703db753d9dbce4decc8154c4b0a5cb069c6c0823171648003bc9f WatchSource:0}: Error finding container 71b6aa07b2703db753d9dbce4decc8154c4b0a5cb069c6c0823171648003bc9f: Status 404 returned error can't find the container with id 71b6aa07b2703db753d9dbce4decc8154c4b0a5cb069c6c0823171648003bc9f
Jul 28 19:11:42.113665 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:42.113631    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt" event={"ID":"322324a7-e80f-456e-b0da-5ddc45698b55","Type":"ContainerStarted","Data":"71b6aa07b2703db753d9dbce4decc8154c4b0a5cb069c6c0823171648003bc9f"}
Jul 28 19:11:43.119874 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:43.119836    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt" event={"ID":"322324a7-e80f-456e-b0da-5ddc45698b55","Type":"ContainerStarted","Data":"ce982751b418dfbb62bca364eda42f523ef2752630f343d06b2a6989fb450735"}
Jul 28 19:11:43.145811 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:43.145758    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt" podStartSLOduration=1.9539271459999998 podStartE2EDuration="3.145739158s" totalImagesPullingTime="1.191812012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:11:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:11:43.14305384 +0000 UTC m=+551.089538721" watchObservedRunningTime="2026-07-28 19:11:43.145739158 +0000 UTC m=+551.092224025"
Jul 28 19:11:51.268178 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:51.268143    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:51.268611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:51.268201    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:51.280202 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:51.280181    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:11:52.168839 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:11:52.168808    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt"
Jul 28 19:12:32.045754 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.045726    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"]
Jul 28 19:12:32.048789 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.048765    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.051629 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.051602    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 28 19:12:32.061549 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.061525    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"]
Jul 28 19:12:32.213838 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.213809    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c71dd818-a008-468d-b733-e23a8c047005-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.213972 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.213846    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c71dd818-a008-468d-b733-e23a8c047005-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.213972 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.213877    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7s9v\" (UniqueName: \"kubernetes.io/projected/c71dd818-a008-468d-b733-e23a8c047005-kube-api-access-d7s9v\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.213972 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.213963    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c71dd818-a008-468d-b733-e23a8c047005-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.214091 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.213990    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c71dd818-a008-468d-b733-e23a8c047005-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.214091 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.214014    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c71dd818-a008-468d-b733-e23a8c047005-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.315267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.315202    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c71dd818-a008-468d-b733-e23a8c047005-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.315267 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.315244    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7s9v\" (UniqueName: \"kubernetes.io/projected/c71dd818-a008-468d-b733-e23a8c047005-kube-api-access-d7s9v\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.315427 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.315275    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c71dd818-a008-468d-b733-e23a8c047005-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.315427 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.315292    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c71dd818-a008-468d-b733-e23a8c047005-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.315427 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.315310    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c71dd818-a008-468d-b733-e23a8c047005-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.315427 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.315352    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c71dd818-a008-468d-b733-e23a8c047005-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.315775 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.315746    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c71dd818-a008-468d-b733-e23a8c047005-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.315892 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.315803    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c71dd818-a008-468d-b733-e23a8c047005-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.315954 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.315932    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c71dd818-a008-468d-b733-e23a8c047005-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.317423 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.317403    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c71dd818-a008-468d-b733-e23a8c047005-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.317707 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.317692    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c71dd818-a008-468d-b733-e23a8c047005-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.323125 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.323099    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7s9v\" (UniqueName: \"kubernetes.io/projected/c71dd818-a008-468d-b733-e23a8c047005-kube-api-access-d7s9v\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-45c2b\" (UID: \"c71dd818-a008-468d-b733-e23a8c047005\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.359375 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.359356    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:32.428530 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.428505    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:12:32.434790 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.434769    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:12:32.435542 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.435518    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:12:32.441637 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.441616    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:12:32.491354 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:32.491335    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"]
Jul 28 19:12:32.493388 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:12:32.493367    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc71dd818_a008_468d_b733_e23a8c047005.slice/crio-19251eef258d559655a19a4cb4c6f997cfb8a962ec5741be847e253a1833279d WatchSource:0}: Error finding container 19251eef258d559655a19a4cb4c6f997cfb8a962ec5741be847e253a1833279d: Status 404 returned error can't find the container with id 19251eef258d559655a19a4cb4c6f997cfb8a962ec5741be847e253a1833279d
Jul 28 19:12:33.343802 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:33.343766    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b" event={"ID":"c71dd818-a008-468d-b733-e23a8c047005","Type":"ContainerStarted","Data":"b88ee8ac6571fb250912b6203ce30fffaeb302adcf551624a9945f0c76f96d9c"}
Jul 28 19:12:33.345854 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:33.343817    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b" event={"ID":"c71dd818-a008-468d-b733-e23a8c047005","Type":"ContainerStarted","Data":"19251eef258d559655a19a4cb4c6f997cfb8a962ec5741be847e253a1833279d"}
Jul 28 19:12:33.361527 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:33.361463    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b" podStartSLOduration=1.162906032 podStartE2EDuration="1.361448685s" totalImagesPullingTime="198.542653ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:12:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:12:33.360979456 +0000 UTC m=+601.307464345" watchObservedRunningTime="2026-07-28 19:12:33.361448685 +0000 UTC m=+601.307933608"
Jul 28 19:12:42.361625 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:42.361587    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:42.362129 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:42.361673    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:42.373827 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:42.373804    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:42.392339 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:42.392313    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-45c2b"
Jul 28 19:12:48.051191 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.051156    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"]
Jul 28 19:12:48.054990 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.054965    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.057772 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.057752    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 28 19:12:48.065599 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.065578    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"]
Jul 28 19:12:48.148449 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.148414    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.148660 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.148461    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.148660 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.148525    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slqgl\" (UniqueName: \"kubernetes.io/projected/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-kube-api-access-slqgl\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.148660 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.148563    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.148660 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.148617    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.148810 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.148673    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.249547 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.249513    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.249547 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.249552    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slqgl\" (UniqueName: \"kubernetes.io/projected/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-kube-api-access-slqgl\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.249801 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.249574    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.249801 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.249591    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.249801 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.249633    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.249801 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.249726    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.250022 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.249998    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.250086 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.250038    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.250123 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.250100    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.251897 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.251875    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.252143 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.252126    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.257970 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.257934    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slqgl\" (UniqueName: \"kubernetes.io/projected/bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d-kube-api-access-slqgl\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lqngh\" (UID: \"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.364772 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.364684    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:48.493572 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.493546    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"]
Jul 28 19:12:48.495692 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:12:48.495660    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbb29a1e_f3f7_4608_ad9e_6e9cb02a377d.slice/crio-a801d4699a885ae212436f2426c2c0b606916b1cba48c489d829a86c0bb83480 WatchSource:0}: Error finding container a801d4699a885ae212436f2426c2c0b606916b1cba48c489d829a86c0bb83480: Status 404 returned error can't find the container with id a801d4699a885ae212436f2426c2c0b606916b1cba48c489d829a86c0bb83480
Jul 28 19:12:48.497165 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:48.497146    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:12:49.413705 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:49.413667    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh" event={"ID":"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d","Type":"ContainerStarted","Data":"b223915f1eee236731afdc3e3f2560d27c2058f87fc9e5bfb537c2f9ef83e876"}
Jul 28 19:12:49.414181 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:49.413715    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh" event={"ID":"bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d","Type":"ContainerStarted","Data":"a801d4699a885ae212436f2426c2c0b606916b1cba48c489d829a86c0bb83480"}
Jul 28 19:12:49.433333 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:49.433288    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh" podStartSLOduration=1.152059087 podStartE2EDuration="1.433270093s" totalImagesPullingTime="281.211006ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:12:49.431216257 +0000 UTC m=+617.377701142" watchObservedRunningTime="2026-07-28 19:12:49.433270093 +0000 UTC m=+617.379754959"
Jul 28 19:12:58.365413 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:58.365319    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:58.365413 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:58.365363    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:58.377525 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:58.377503    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:12:58.465120 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:12:58.465085    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lqngh"
Jul 28 19:13:57.438001 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:13:57.437972    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8694dc4c86-q47rl"]
Jul 28 19:13:57.441157 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:13:57.441141    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8694dc4c86-q47rl"
Jul 28 19:13:57.445037 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:13:57.445012    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-r8kdh\""
Jul 28 19:13:57.445037 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:13:57.445020    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 28 19:13:57.449429 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:13:57.449410    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8694dc4c86-q47rl"]
Jul 28 19:13:57.515959 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:13:57.515929    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/22468a84-6db3-48b0-911f-b2d0cfd2862f-tls-cert\") pod \"authorino-8694dc4c86-q47rl\" (UID: \"22468a84-6db3-48b0-911f-b2d0cfd2862f\") " pod="kuadrant-system/authorino-8694dc4c86-q47rl"
Jul 28 19:13:57.515959 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:13:57.515961    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gbww\" (UniqueName: \"kubernetes.io/projected/22468a84-6db3-48b0-911f-b2d0cfd2862f-kube-api-access-2gbww\") pod \"authorino-8694dc4c86-q47rl\" (UID: \"22468a84-6db3-48b0-911f-b2d0cfd2862f\") " pod="kuadrant-system/authorino-8694dc4c86-q47rl"
Jul 28 19:13:57.616717 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:13:57.616690    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/22468a84-6db3-48b0-911f-b2d0cfd2862f-tls-cert\") pod \"authorino-8694dc4c86-q47rl\" (UID: \"22468a84-6db3-48b0-911f-b2d0cfd2862f\") " pod="kuadrant-system/authorino-8694dc4c86-q47rl"
Jul 28 19:13:57.616717 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:13:57.616717    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gbww\" (UniqueName: \"kubernetes.io/projected/22468a84-6db3-48b0-911f-b2d0cfd2862f-kube-api-access-2gbww\") pod \"authorino-8694dc4c86-q47rl\" (UID: \"22468a84-6db3-48b0-911f-b2d0cfd2862f\") " pod="kuadrant-system/authorino-8694dc4c86-q47rl"
Jul 28 19:13:57.619077 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:13:57.619057    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/22468a84-6db3-48b0-911f-b2d0cfd2862f-tls-cert\") pod \"authorino-8694dc4c86-q47rl\" (UID: \"22468a84-6db3-48b0-911f-b2d0cfd2862f\") " pod="kuadrant-system/authorino-8694dc4c86-q47rl"
Jul 28 19:13:57.624420 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:13:57.624403    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gbww\" (UniqueName: \"kubernetes.io/projected/22468a84-6db3-48b0-911f-b2d0cfd2862f-kube-api-access-2gbww\") pod \"authorino-8694dc4c86-q47rl\" (UID: \"22468a84-6db3-48b0-911f-b2d0cfd2862f\") " pod="kuadrant-system/authorino-8694dc4c86-q47rl"
Jul 28 19:13:57.750429 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:13:57.750401    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8694dc4c86-q47rl"
Jul 28 19:13:57.904111 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:13:57.904079    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8694dc4c86-q47rl"]
Jul 28 19:13:57.905457 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:13:57.905430    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22468a84_6db3_48b0_911f_b2d0cfd2862f.slice/crio-36fca6adc93af1c5d99fa2af62fe69b2218675e236dcee2d6b434c88f5d22d30 WatchSource:0}: Error finding container 36fca6adc93af1c5d99fa2af62fe69b2218675e236dcee2d6b434c88f5d22d30: Status 404 returned error can't find the container with id 36fca6adc93af1c5d99fa2af62fe69b2218675e236dcee2d6b434c88f5d22d30
Jul 28 19:13:58.726766 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:13:58.726732    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8694dc4c86-q47rl" event={"ID":"22468a84-6db3-48b0-911f-b2d0cfd2862f","Type":"ContainerStarted","Data":"4d0a64286bdaa27e3b5e2b7f0bfdc0775ee12862500f45315b435f8526de06a2"}
Jul 28 19:13:58.726766 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:13:58.726774    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8694dc4c86-q47rl" event={"ID":"22468a84-6db3-48b0-911f-b2d0cfd2862f","Type":"ContainerStarted","Data":"36fca6adc93af1c5d99fa2af62fe69b2218675e236dcee2d6b434c88f5d22d30"}
Jul 28 19:13:58.750322 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:13:58.750270    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8694dc4c86-q47rl" podStartSLOduration=1.323272118 podStartE2EDuration="1.750254689s" totalImagesPullingTime="426.982571ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:13:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:13:58.749435451 +0000 UTC m=+686.695920318" watchObservedRunningTime="2026-07-28 19:13:58.750254689 +0000 UTC m=+686.696739557"
Jul 28 19:17:32.459885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:17:32.459857    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:17:32.465925 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:17:32.465903    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:17:32.469202 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:17:32.469179    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:17:32.472606 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:17:32.472590    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:21:11.309127 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:11.309092    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"]
Jul 28 19:21:11.309753 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:11.309311    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px" podUID="605020fe-c660-4804-869e-3e1c23d5cf88" containerName="manager" containerID="cri-o://0bebb009ae154c65e461359be90ccdbca575e263fff18cb4ef46c899bcb7f0fc" gracePeriod=10
Jul 28 19:21:14.685314 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:14.685237    2561 generic.go:356] "Generic (PLEG): container finished" podID="605020fe-c660-4804-869e-3e1c23d5cf88" containerID="0bebb009ae154c65e461359be90ccdbca575e263fff18cb4ef46c899bcb7f0fc" exitCode=0
Jul 28 19:21:14.685314 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:14.685288    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px" event={"ID":"605020fe-c660-4804-869e-3e1c23d5cf88","Type":"ContainerDied","Data":"0bebb009ae154c65e461359be90ccdbca575e263fff18cb4ef46c899bcb7f0fc"}
Jul 28 19:21:14.854262 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:14.854238    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"
Jul 28 19:21:14.992040 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:14.992007    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/605020fe-c660-4804-869e-3e1c23d5cf88-extensions-socket-volume\") pod \"605020fe-c660-4804-869e-3e1c23d5cf88\" (UID: \"605020fe-c660-4804-869e-3e1c23d5cf88\") "
Jul 28 19:21:14.992040 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:14.992045    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s55h5\" (UniqueName: \"kubernetes.io/projected/605020fe-c660-4804-869e-3e1c23d5cf88-kube-api-access-s55h5\") pod \"605020fe-c660-4804-869e-3e1c23d5cf88\" (UID: \"605020fe-c660-4804-869e-3e1c23d5cf88\") "
Jul 28 19:21:14.992379 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:14.992356    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/605020fe-c660-4804-869e-3e1c23d5cf88-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "605020fe-c660-4804-869e-3e1c23d5cf88" (UID: "605020fe-c660-4804-869e-3e1c23d5cf88"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:21:14.994018 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:14.993995    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/605020fe-c660-4804-869e-3e1c23d5cf88-kube-api-access-s55h5" (OuterVolumeSpecName: "kube-api-access-s55h5") pod "605020fe-c660-4804-869e-3e1c23d5cf88" (UID: "605020fe-c660-4804-869e-3e1c23d5cf88"). InnerVolumeSpecName "kube-api-access-s55h5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:21:15.093475 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:15.093447    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/605020fe-c660-4804-869e-3e1c23d5cf88-extensions-socket-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:21:15.093475 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:15.093471    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s55h5\" (UniqueName: \"kubernetes.io/projected/605020fe-c660-4804-869e-3e1c23d5cf88-kube-api-access-s55h5\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:21:15.691302 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:15.691273    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"
Jul 28 19:21:15.691738 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:15.691303    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px" event={"ID":"605020fe-c660-4804-869e-3e1c23d5cf88","Type":"ContainerDied","Data":"34e4c4c5d4381ea95f568131516f9c180c24ad61412327cf37909f0d23457bf8"}
Jul 28 19:21:15.691738 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:15.691349    2561 scope.go:117] "RemoveContainer" containerID="0bebb009ae154c65e461359be90ccdbca575e263fff18cb4ef46c899bcb7f0fc"
Jul 28 19:21:15.715061 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:15.715040    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"]
Jul 28 19:21:15.718658 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:15.718635    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g88px"]
Jul 28 19:21:16.603985 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:21:16.603948    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="605020fe-c660-4804-869e-3e1c23d5cf88" path="/var/lib/kubelet/pods/605020fe-c660-4804-869e-3e1c23d5cf88/volumes"
Jul 28 19:22:32.492718 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:22:32.492638    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:22:32.493256 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:22:32.493239    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:22:32.498728 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:22:32.498706    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:22:32.499584 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:22:32.499561    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:25:16.762715 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.762633    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"]
Jul 28 19:25:16.765292 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.763018    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="605020fe-c660-4804-869e-3e1c23d5cf88" containerName="manager"
Jul 28 19:25:16.765292 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.763032    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="605020fe-c660-4804-869e-3e1c23d5cf88" containerName="manager"
Jul 28 19:25:16.765292 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.763136    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="605020fe-c660-4804-869e-3e1c23d5cf88" containerName="manager"
Jul 28 19:25:16.766269 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.766245    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.768838 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.768807    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-9c7e9420-kserve-self-signed-certs\""
Jul 28 19:25:16.786577 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.786555    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"]
Jul 28 19:25:16.853091 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.853062    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpn7f\" (UniqueName: \"kubernetes.io/projected/f0036a09-64ef-4e0c-bedb-d160815e93ab-kube-api-access-wpn7f\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.853181 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.853097    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0036a09-64ef-4e0c-bedb-d160815e93ab-tls-certs\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.853181 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.853127    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-home\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.853181 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.853169    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-model-cache\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.853313 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.853277    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-tmp-dir\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.853369 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.853351    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-dshm\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.954232 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.954204    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpn7f\" (UniqueName: \"kubernetes.io/projected/f0036a09-64ef-4e0c-bedb-d160815e93ab-kube-api-access-wpn7f\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.954348 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.954240    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0036a09-64ef-4e0c-bedb-d160815e93ab-tls-certs\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.954348 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.954285    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-home\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.954348 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.954308    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-model-cache\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.954539 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.954363    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-tmp-dir\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.954539 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.954430    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-dshm\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.954724 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.954700    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-model-cache\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.954787 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.954758    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-home\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.954904 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.954806    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-tmp-dir\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.956693 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.956669    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-dshm\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.956825 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.956806    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0036a09-64ef-4e0c-bedb-d160815e93ab-tls-certs\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:16.962768 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:16.962750    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpn7f\" (UniqueName: \"kubernetes.io/projected/f0036a09-64ef-4e0c-bedb-d160815e93ab-kube-api-access-wpn7f\") pod \"e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") " pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:17.077172 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:17.077118    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:17.212858 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:17.212822    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"]
Jul 28 19:25:17.215016 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:25:17.214992    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0036a09_64ef_4e0c_bedb_d160815e93ab.slice/crio-971b2add2ee25f70732ccd7fa1c39336543d64d427e63dfe152d54ea238471fa WatchSource:0}: Error finding container 971b2add2ee25f70732ccd7fa1c39336543d64d427e63dfe152d54ea238471fa: Status 404 returned error can't find the container with id 971b2add2ee25f70732ccd7fa1c39336543d64d427e63dfe152d54ea238471fa
Jul 28 19:25:17.826317 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:17.826280    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd" event={"ID":"f0036a09-64ef-4e0c-bedb-d160815e93ab","Type":"ContainerStarted","Data":"156020a8a0c2b4b7df6c7ea247e1e58a6a26d440439c94914d7d0a8244ec851e"}
Jul 28 19:25:17.826317 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:17.826331    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd" event={"ID":"f0036a09-64ef-4e0c-bedb-d160815e93ab","Type":"ContainerStarted","Data":"971b2add2ee25f70732ccd7fa1c39336543d64d427e63dfe152d54ea238471fa"}
Jul 28 19:25:26.975456 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:26.975378    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd" podStartSLOduration=10.975350852 podStartE2EDuration="10.975350852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:25:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:25:17.853980319 +0000 UTC m=+1365.800465187" watchObservedRunningTime="2026-07-28 19:25:26.975350852 +0000 UTC m=+1374.921835721"
Jul 28 19:25:26.975992 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:26.975948    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"]
Jul 28 19:25:26.979678 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:26.979655    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:26.984022 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:26.984002    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-9c7e9420-kserve-self-signed-certs\""
Jul 28 19:25:27.001130 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.001068    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"]
Jul 28 19:25:27.035819 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.035798    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-tmp-dir\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.035953 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.035836    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01358ac5-373c-4c55-b3f4-b5e15b4aba71-tls-certs\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.035953 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.035931    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-home\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.036065 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.035972    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnnkh\" (UniqueName: \"kubernetes.io/projected/01358ac5-373c-4c55-b3f4-b5e15b4aba71-kube-api-access-pnnkh\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.036065 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.036047    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-dshm\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.036148 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.036071    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-model-cache\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.077542 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.077521    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:27.077644 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.077558    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:27.089643 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.089625    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:27.136677 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.136654    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-home\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.136775 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.136681    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnnkh\" (UniqueName: \"kubernetes.io/projected/01358ac5-373c-4c55-b3f4-b5e15b4aba71-kube-api-access-pnnkh\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.136775 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.136761    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-dshm\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.136888 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.136802    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-model-cache\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.136888 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.136848    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-tmp-dir\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.136888 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.136876    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01358ac5-373c-4c55-b3f4-b5e15b4aba71-tls-certs\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.137085 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.137062    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-home\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.137193 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.137176    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-model-cache\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.137285 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.137267    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-tmp-dir\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.138915 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.138894    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-dshm\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.139240 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.139223    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01358ac5-373c-4c55-b3f4-b5e15b4aba71-tls-certs\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.146820 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.146801    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnnkh\" (UniqueName: \"kubernetes.io/projected/01358ac5-373c-4c55-b3f4-b5e15b4aba71-kube-api-access-pnnkh\") pod \"e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") " pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.290167 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.290111    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:27.424375 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.424353    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"]
Jul 28 19:25:27.427249 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:25:27.427220    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01358ac5_373c_4c55_b3f4_b5e15b4aba71.slice/crio-b219337476e4c30312ca927eec2faaf3bd5b69238b6914003538a5abe5f7c175 WatchSource:0}: Error finding container b219337476e4c30312ca927eec2faaf3bd5b69238b6914003538a5abe5f7c175: Status 404 returned error can't find the container with id b219337476e4c30312ca927eec2faaf3bd5b69238b6914003538a5abe5f7c175
Jul 28 19:25:27.879003 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.878966    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf" event={"ID":"01358ac5-373c-4c55-b3f4-b5e15b4aba71","Type":"ContainerStarted","Data":"4a8396d2626e6c6a4dd035286deecb4f018fd6ba47c1de674631f26f7e4a3ad9"}
Jul 28 19:25:27.879170 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.879017    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf" event={"ID":"01358ac5-373c-4c55-b3f4-b5e15b4aba71","Type":"ContainerStarted","Data":"b219337476e4c30312ca927eec2faaf3bd5b69238b6914003538a5abe5f7c175"}
Jul 28 19:25:27.890786 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.890757    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:25:27.900678 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:27.900438    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf" podStartSLOduration=1.900417214 podStartE2EDuration="1.900417214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:25:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:25:27.899253514 +0000 UTC m=+1375.845738381" watchObservedRunningTime="2026-07-28 19:25:27.900417214 +0000 UTC m=+1375.846902083"
Jul 28 19:25:37.290584 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:37.290544    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:37.291021 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:37.290602    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:37.302815 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:37.302791    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:25:37.942506 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:25:37.942457    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:26:01.538096 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:01.538059    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"]
Jul 28 19:26:02.536974 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:02.536941    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"]
Jul 28 19:26:16.557898 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.557858    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd" podUID="f0036a09-64ef-4e0c-bedb-d160815e93ab" containerName="main" containerID="cri-o://156020a8a0c2b4b7df6c7ea247e1e58a6a26d440439c94914d7d0a8244ec851e" gracePeriod=45
Jul 28 19:26:16.806110 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.806085    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:26:16.855041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.854981    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-tmp-dir\") pod \"f0036a09-64ef-4e0c-bedb-d160815e93ab\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") "
Jul 28 19:26:16.855141 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.855045    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpn7f\" (UniqueName: \"kubernetes.io/projected/f0036a09-64ef-4e0c-bedb-d160815e93ab-kube-api-access-wpn7f\") pod \"f0036a09-64ef-4e0c-bedb-d160815e93ab\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") "
Jul 28 19:26:16.855141 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.855062    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-dshm\") pod \"f0036a09-64ef-4e0c-bedb-d160815e93ab\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") "
Jul 28 19:26:16.855141 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.855084    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-home\") pod \"f0036a09-64ef-4e0c-bedb-d160815e93ab\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") "
Jul 28 19:26:16.855241 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.855218    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f0036a09-64ef-4e0c-bedb-d160815e93ab" (UID: "f0036a09-64ef-4e0c-bedb-d160815e93ab"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:26:16.855276 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.855266    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-model-cache\") pod \"f0036a09-64ef-4e0c-bedb-d160815e93ab\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") "
Jul 28 19:26:16.855333 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.855318    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0036a09-64ef-4e0c-bedb-d160815e93ab-tls-certs\") pod \"f0036a09-64ef-4e0c-bedb-d160815e93ab\" (UID: \"f0036a09-64ef-4e0c-bedb-d160815e93ab\") "
Jul 28 19:26:16.855377 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.855343    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-home" (OuterVolumeSpecName: "home") pod "f0036a09-64ef-4e0c-bedb-d160815e93ab" (UID: "f0036a09-64ef-4e0c-bedb-d160815e93ab"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:26:16.855550 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.855526    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-model-cache" (OuterVolumeSpecName: "model-cache") pod "f0036a09-64ef-4e0c-bedb-d160815e93ab" (UID: "f0036a09-64ef-4e0c-bedb-d160815e93ab"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:26:16.855696 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.855682    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-model-cache\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:26:16.855764 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.855700    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-tmp-dir\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:26:16.855764 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.855712    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-home\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:26:16.857363 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.857331    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0036a09-64ef-4e0c-bedb-d160815e93ab-kube-api-access-wpn7f" (OuterVolumeSpecName: "kube-api-access-wpn7f") pod "f0036a09-64ef-4e0c-bedb-d160815e93ab" (UID: "f0036a09-64ef-4e0c-bedb-d160815e93ab"). InnerVolumeSpecName "kube-api-access-wpn7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:26:16.857363 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.857350    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0036a09-64ef-4e0c-bedb-d160815e93ab-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f0036a09-64ef-4e0c-bedb-d160815e93ab" (UID: "f0036a09-64ef-4e0c-bedb-d160815e93ab"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:26:16.857502 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.857333    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-dshm" (OuterVolumeSpecName: "dshm") pod "f0036a09-64ef-4e0c-bedb-d160815e93ab" (UID: "f0036a09-64ef-4e0c-bedb-d160815e93ab"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:26:16.956972 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.956949    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0036a09-64ef-4e0c-bedb-d160815e93ab-tls-certs\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:26:16.956972 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.956971    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpn7f\" (UniqueName: \"kubernetes.io/projected/f0036a09-64ef-4e0c-bedb-d160815e93ab-kube-api-access-wpn7f\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:26:16.957099 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:16.956980    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0036a09-64ef-4e0c-bedb-d160815e93ab-dshm\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:26:17.119834 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.119758    2561 generic.go:356] "Generic (PLEG): container finished" podID="f0036a09-64ef-4e0c-bedb-d160815e93ab" containerID="156020a8a0c2b4b7df6c7ea247e1e58a6a26d440439c94914d7d0a8244ec851e" exitCode=0
Jul 28 19:26:17.119834 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.119818    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"
Jul 28 19:26:17.120042 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.119844    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd" event={"ID":"f0036a09-64ef-4e0c-bedb-d160815e93ab","Type":"ContainerDied","Data":"156020a8a0c2b4b7df6c7ea247e1e58a6a26d440439c94914d7d0a8244ec851e"}
Jul 28 19:26:17.120042 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.119893    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd" event={"ID":"f0036a09-64ef-4e0c-bedb-d160815e93ab","Type":"ContainerDied","Data":"971b2add2ee25f70732ccd7fa1c39336543d64d427e63dfe152d54ea238471fa"}
Jul 28 19:26:17.120042 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.119922    2561 scope.go:117] "RemoveContainer" containerID="156020a8a0c2b4b7df6c7ea247e1e58a6a26d440439c94914d7d0a8244ec851e"
Jul 28 19:26:17.129051 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.129033    2561 scope.go:117] "RemoveContainer" containerID="156020a8a0c2b4b7df6c7ea247e1e58a6a26d440439c94914d7d0a8244ec851e"
Jul 28 19:26:17.129302 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:26:17.129285    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"156020a8a0c2b4b7df6c7ea247e1e58a6a26d440439c94914d7d0a8244ec851e\": container with ID starting with 156020a8a0c2b4b7df6c7ea247e1e58a6a26d440439c94914d7d0a8244ec851e not found: ID does not exist" containerID="156020a8a0c2b4b7df6c7ea247e1e58a6a26d440439c94914d7d0a8244ec851e"
Jul 28 19:26:17.129350 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.129310    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"156020a8a0c2b4b7df6c7ea247e1e58a6a26d440439c94914d7d0a8244ec851e"} err="failed to get container status \"156020a8a0c2b4b7df6c7ea247e1e58a6a26d440439c94914d7d0a8244ec851e\": rpc error: code = NotFound desc = could not find container \"156020a8a0c2b4b7df6c7ea247e1e58a6a26d440439c94914d7d0a8244ec851e\": container with ID starting with 156020a8a0c2b4b7df6c7ea247e1e58a6a26d440439c94914d7d0a8244ec851e not found: ID does not exist"
Jul 28 19:26:17.141516 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.141477    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"]
Jul 28 19:26:17.151111 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.151090    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-9c7e9420-kserve-5f457cc747-xp7kd"]
Jul 28 19:26:17.555364 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.555329    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf" podUID="01358ac5-373c-4c55-b3f4-b5e15b4aba71" containerName="main" containerID="cri-o://4a8396d2626e6c6a4dd035286deecb4f018fd6ba47c1de674631f26f7e4a3ad9" gracePeriod=45
Jul 28 19:26:17.811802 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.811751    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:26:17.863969 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.863947    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01358ac5-373c-4c55-b3f4-b5e15b4aba71-tls-certs\") pod \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") "
Jul 28 19:26:17.864072 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.863997    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-dshm\") pod \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") "
Jul 28 19:26:17.864072 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.864036    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-home\") pod \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") "
Jul 28 19:26:17.864145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.864106    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-model-cache\") pod \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") "
Jul 28 19:26:17.864198 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.864184    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-tmp-dir\") pod \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") "
Jul 28 19:26:17.864254 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.864239    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnnkh\" (UniqueName: \"kubernetes.io/projected/01358ac5-373c-4c55-b3f4-b5e15b4aba71-kube-api-access-pnnkh\") pod \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\" (UID: \"01358ac5-373c-4c55-b3f4-b5e15b4aba71\") "
Jul 28 19:26:17.864323 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.864298    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-home" (OuterVolumeSpecName: "home") pod "01358ac5-373c-4c55-b3f4-b5e15b4aba71" (UID: "01358ac5-373c-4c55-b3f4-b5e15b4aba71"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:26:17.864471 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.864444    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "01358ac5-373c-4c55-b3f4-b5e15b4aba71" (UID: "01358ac5-373c-4c55-b3f4-b5e15b4aba71"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:26:17.864471 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.864455    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-model-cache" (OuterVolumeSpecName: "model-cache") pod "01358ac5-373c-4c55-b3f4-b5e15b4aba71" (UID: "01358ac5-373c-4c55-b3f4-b5e15b4aba71"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:26:17.864615 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.864597    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-home\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:26:17.864652 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.864621    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-model-cache\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:26:17.864652 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.864635    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-tmp-dir\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:26:17.866242 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.866220    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01358ac5-373c-4c55-b3f4-b5e15b4aba71-kube-api-access-pnnkh" (OuterVolumeSpecName: "kube-api-access-pnnkh") pod "01358ac5-373c-4c55-b3f4-b5e15b4aba71" (UID: "01358ac5-373c-4c55-b3f4-b5e15b4aba71"). InnerVolumeSpecName "kube-api-access-pnnkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:26:17.866538 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.866504    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01358ac5-373c-4c55-b3f4-b5e15b4aba71-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "01358ac5-373c-4c55-b3f4-b5e15b4aba71" (UID: "01358ac5-373c-4c55-b3f4-b5e15b4aba71"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:26:17.866538 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.866502    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-dshm" (OuterVolumeSpecName: "dshm") pod "01358ac5-373c-4c55-b3f4-b5e15b4aba71" (UID: "01358ac5-373c-4c55-b3f4-b5e15b4aba71"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:26:17.965641 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.965621    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01358ac5-373c-4c55-b3f4-b5e15b4aba71-dshm\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:26:17.965641 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.965640    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnnkh\" (UniqueName: \"kubernetes.io/projected/01358ac5-373c-4c55-b3f4-b5e15b4aba71-kube-api-access-pnnkh\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:26:17.965766 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:17.965651    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01358ac5-373c-4c55-b3f4-b5e15b4aba71-tls-certs\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:26:18.126053 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:18.125995    2561 generic.go:356] "Generic (PLEG): container finished" podID="01358ac5-373c-4c55-b3f4-b5e15b4aba71" containerID="4a8396d2626e6c6a4dd035286deecb4f018fd6ba47c1de674631f26f7e4a3ad9" exitCode=0
Jul 28 19:26:18.126053 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:18.126044    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"
Jul 28 19:26:18.126208 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:18.126082    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf" event={"ID":"01358ac5-373c-4c55-b3f4-b5e15b4aba71","Type":"ContainerDied","Data":"4a8396d2626e6c6a4dd035286deecb4f018fd6ba47c1de674631f26f7e4a3ad9"}
Jul 28 19:26:18.126208 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:18.126124    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf" event={"ID":"01358ac5-373c-4c55-b3f4-b5e15b4aba71","Type":"ContainerDied","Data":"b219337476e4c30312ca927eec2faaf3bd5b69238b6914003538a5abe5f7c175"}
Jul 28 19:26:18.126208 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:18.126149    2561 scope.go:117] "RemoveContainer" containerID="4a8396d2626e6c6a4dd035286deecb4f018fd6ba47c1de674631f26f7e4a3ad9"
Jul 28 19:26:18.135518 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:18.135470    2561 scope.go:117] "RemoveContainer" containerID="4a8396d2626e6c6a4dd035286deecb4f018fd6ba47c1de674631f26f7e4a3ad9"
Jul 28 19:26:18.135797 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:26:18.135776    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a8396d2626e6c6a4dd035286deecb4f018fd6ba47c1de674631f26f7e4a3ad9\": container with ID starting with 4a8396d2626e6c6a4dd035286deecb4f018fd6ba47c1de674631f26f7e4a3ad9 not found: ID does not exist" containerID="4a8396d2626e6c6a4dd035286deecb4f018fd6ba47c1de674631f26f7e4a3ad9"
Jul 28 19:26:18.135845 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:18.135807    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a8396d2626e6c6a4dd035286deecb4f018fd6ba47c1de674631f26f7e4a3ad9"} err="failed to get container status \"4a8396d2626e6c6a4dd035286deecb4f018fd6ba47c1de674631f26f7e4a3ad9\": rpc error: code = NotFound desc = could not find container \"4a8396d2626e6c6a4dd035286deecb4f018fd6ba47c1de674631f26f7e4a3ad9\": container with ID starting with 4a8396d2626e6c6a4dd035286deecb4f018fd6ba47c1de674631f26f7e4a3ad9 not found: ID does not exist"
Jul 28 19:26:18.147935 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:18.147909    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"]
Jul 28 19:26:18.153736 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:18.153711    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-9c7e9420-kserve-7c6564bbc-mqftf"]
Jul 28 19:26:18.603567 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:18.603541    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01358ac5-373c-4c55-b3f4-b5e15b4aba71" path="/var/lib/kubelet/pods/01358ac5-373c-4c55-b3f4-b5e15b4aba71/volumes"
Jul 28 19:26:18.603930 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:26:18.603917    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0036a09-64ef-4e0c-bedb-d160815e93ab" path="/var/lib/kubelet/pods/f0036a09-64ef-4e0c-bedb-d160815e93ab/volumes"
Jul 28 19:27:32.522277 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:27:32.522248    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:27:32.523117 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:27:32.523093    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:27:32.528652 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:27:32.528629    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:27:32.529395 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:27:32.529371    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:29:52.451434 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.451396    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"]
Jul 28 19:29:52.452027 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.451882    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0036a09-64ef-4e0c-bedb-d160815e93ab" containerName="main"
Jul 28 19:29:52.452027 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.451898    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0036a09-64ef-4e0c-bedb-d160815e93ab" containerName="main"
Jul 28 19:29:52.452027 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.451910    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01358ac5-373c-4c55-b3f4-b5e15b4aba71" containerName="main"
Jul 28 19:29:52.452027 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.451916    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="01358ac5-373c-4c55-b3f4-b5e15b4aba71" containerName="main"
Jul 28 19:29:52.452027 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.451989    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="01358ac5-373c-4c55-b3f4-b5e15b4aba71" containerName="main"
Jul 28 19:29:52.452027 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.451995    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0036a09-64ef-4e0c-bedb-d160815e93ab" containerName="main"
Jul 28 19:29:52.455452 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.455429    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"
Jul 28 19:29:52.457912 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.457888    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-16c9dd30-serving-cert\""
Jul 28 19:29:52.458358 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.458340    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 19:29:52.458938 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.458922    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:29:52.459036 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.458925    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z259w\""
Jul 28 19:29:52.465068 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.465041    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"]
Jul 28 19:29:52.590720 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.590664    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp2m2\" (UniqueName: \"kubernetes.io/projected/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-kube-api-access-xp2m2\") pod \"maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z\" (UID: \"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"
Jul 28 19:29:52.590720 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.590724    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-maas-api-tls\") pod \"maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z\" (UID: \"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"
Jul 28 19:29:52.590951 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.590801    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-openshift-service-ca\") pod \"maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z\" (UID: \"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"
Jul 28 19:29:52.691618 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.691579    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp2m2\" (UniqueName: \"kubernetes.io/projected/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-kube-api-access-xp2m2\") pod \"maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z\" (UID: \"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"
Jul 28 19:29:52.691618 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.691624    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-maas-api-tls\") pod \"maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z\" (UID: \"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"
Jul 28 19:29:52.691860 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.691661    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-openshift-service-ca\") pod \"maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z\" (UID: \"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"
Jul 28 19:29:52.692364 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.692339    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-openshift-service-ca\") pod \"maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z\" (UID: \"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"
Jul 28 19:29:52.694073 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.694055    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-maas-api-tls\") pod \"maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z\" (UID: \"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"
Jul 28 19:29:52.701764 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.701702    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp2m2\" (UniqueName: \"kubernetes.io/projected/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-kube-api-access-xp2m2\") pod \"maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z\" (UID: \"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"
Jul 28 19:29:52.767260 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.767228    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"
Jul 28 19:29:52.911990 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.911901    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"]
Jul 28 19:29:52.914192 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:29:52.914164    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb5c0383_fd1d_4f5e_9eb3_9d6300323c2b.slice/crio-b8ca0817cab799d6726347a21a915ecfd5a81e97bda72a319e5c98bcf7d6a153 WatchSource:0}: Error finding container b8ca0817cab799d6726347a21a915ecfd5a81e97bda72a319e5c98bcf7d6a153: Status 404 returned error can't find the container with id b8ca0817cab799d6726347a21a915ecfd5a81e97bda72a319e5c98bcf7d6a153
Jul 28 19:29:52.915427 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:52.915411    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:29:53.140595 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.140517    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z" event={"ID":"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b","Type":"ContainerStarted","Data":"b8ca0817cab799d6726347a21a915ecfd5a81e97bda72a319e5c98bcf7d6a153"}
Jul 28 19:29:53.211970 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.211939    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"]
Jul 28 19:29:53.216458 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.216440    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"
Jul 28 19:29:53.219054 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.219034    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-16c9dd30-dockercfg-nnq5m\""
Jul 28 19:29:53.219383 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.219366    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-16c9dd30\""
Jul 28 19:29:53.228685 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.228667    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"]
Jul 28 19:29:53.246224 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.246203    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"]
Jul 28 19:29:53.249602 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.249579    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"
Jul 28 19:29:53.264643 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.264622    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"]
Jul 28 19:29:53.399346 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.399278    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhr5j\" (UniqueName: \"kubernetes.io/projected/029a8673-2c24-4f12-b98b-2f7f3a9a50c1-kube-api-access-lhr5j\") pod \"payload-processing-e2e-ait-16c9dd30-748588d576-brcd8\" (UID: \"029a8673-2c24-4f12-b98b-2f7f3a9a50c1\") " pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"
Jul 28 19:29:53.399346 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.399319    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vldlm\" (UniqueName: \"kubernetes.io/projected/09a424a6-93be-4313-91fa-a193943418ac-kube-api-access-vldlm\") pod \"payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn\" (UID: \"09a424a6-93be-4313-91fa-a193943418ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"
Jul 28 19:29:53.399535 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.399383    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/029a8673-2c24-4f12-b98b-2f7f3a9a50c1-plugins-config-volume\") pod \"payload-processing-e2e-ait-16c9dd30-748588d576-brcd8\" (UID: \"029a8673-2c24-4f12-b98b-2f7f3a9a50c1\") " pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"
Jul 28 19:29:53.399535 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.399400    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09a424a6-93be-4313-91fa-a193943418ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn\" (UID: \"09a424a6-93be-4313-91fa-a193943418ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"
Jul 28 19:29:53.500406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.500368    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhr5j\" (UniqueName: \"kubernetes.io/projected/029a8673-2c24-4f12-b98b-2f7f3a9a50c1-kube-api-access-lhr5j\") pod \"payload-processing-e2e-ait-16c9dd30-748588d576-brcd8\" (UID: \"029a8673-2c24-4f12-b98b-2f7f3a9a50c1\") " pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"
Jul 28 19:29:53.500911 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.500449    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vldlm\" (UniqueName: \"kubernetes.io/projected/09a424a6-93be-4313-91fa-a193943418ac-kube-api-access-vldlm\") pod \"payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn\" (UID: \"09a424a6-93be-4313-91fa-a193943418ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"
Jul 28 19:29:53.500911 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.500559    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/029a8673-2c24-4f12-b98b-2f7f3a9a50c1-plugins-config-volume\") pod \"payload-processing-e2e-ait-16c9dd30-748588d576-brcd8\" (UID: \"029a8673-2c24-4f12-b98b-2f7f3a9a50c1\") " pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"
Jul 28 19:29:53.500911 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.500591    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09a424a6-93be-4313-91fa-a193943418ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn\" (UID: \"09a424a6-93be-4313-91fa-a193943418ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"
Jul 28 19:29:53.501601 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.501571    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09a424a6-93be-4313-91fa-a193943418ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn\" (UID: \"09a424a6-93be-4313-91fa-a193943418ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"
Jul 28 19:29:53.501601 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.501571    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/029a8673-2c24-4f12-b98b-2f7f3a9a50c1-plugins-config-volume\") pod \"payload-processing-e2e-ait-16c9dd30-748588d576-brcd8\" (UID: \"029a8673-2c24-4f12-b98b-2f7f3a9a50c1\") " pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"
Jul 28 19:29:53.509618 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.509590    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vldlm\" (UniqueName: \"kubernetes.io/projected/09a424a6-93be-4313-91fa-a193943418ac-kube-api-access-vldlm\") pod \"payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn\" (UID: \"09a424a6-93be-4313-91fa-a193943418ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"
Jul 28 19:29:53.509752 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.509637    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhr5j\" (UniqueName: \"kubernetes.io/projected/029a8673-2c24-4f12-b98b-2f7f3a9a50c1-kube-api-access-lhr5j\") pod \"payload-processing-e2e-ait-16c9dd30-748588d576-brcd8\" (UID: \"029a8673-2c24-4f12-b98b-2f7f3a9a50c1\") " pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"
Jul 28 19:29:53.527330 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.527305    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"
Jul 28 19:29:53.559888 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.559611    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"
Jul 28 19:29:53.715556 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.714641    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"]
Jul 28 19:29:53.747817 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:53.747794    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"]
Jul 28 19:29:53.751036 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:29:53.751005    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029a8673_2c24_4f12_b98b_2f7f3a9a50c1.slice/crio-542e1249bc794eec7d96cf482681f1a7ece5d1c18098f1bea30e1c362dcc2635 WatchSource:0}: Error finding container 542e1249bc794eec7d96cf482681f1a7ece5d1c18098f1bea30e1c362dcc2635: Status 404 returned error can't find the container with id 542e1249bc794eec7d96cf482681f1a7ece5d1c18098f1bea30e1c362dcc2635
Jul 28 19:29:54.147901 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:54.147760    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8" event={"ID":"029a8673-2c24-4f12-b98b-2f7f3a9a50c1","Type":"ContainerStarted","Data":"9c97b6fcfb8907d952643123842485df8e6e078b56fcc8f08887becd9a7fc75a"}
Jul 28 19:29:54.147901 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:54.147829    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"
Jul 28 19:29:54.147901 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:54.147857    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8" event={"ID":"029a8673-2c24-4f12-b98b-2f7f3a9a50c1","Type":"ContainerStarted","Data":"542e1249bc794eec7d96cf482681f1a7ece5d1c18098f1bea30e1c362dcc2635"}
Jul 28 19:29:54.156325 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:54.156295    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn" event={"ID":"09a424a6-93be-4313-91fa-a193943418ac","Type":"ContainerStarted","Data":"bc927d76daa2dcc323a953ec06f1b420d9a46baa58921501b264b46eb6714378"}
Jul 28 19:29:54.156460 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:54.156374    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn" event={"ID":"09a424a6-93be-4313-91fa-a193943418ac","Type":"ContainerStarted","Data":"07d34ae703c35d1453637ce4e1153a0175763f9943c8faf0d4c5a224872002f8"}
Jul 28 19:29:54.156460 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:54.156418    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"
Jul 28 19:29:54.167996 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:54.167946    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8" podStartSLOduration=1.167926432 podStartE2EDuration="1.167926432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:29:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:29:54.165516127 +0000 UTC m=+1642.112001021" watchObservedRunningTime="2026-07-28 19:29:54.167926432 +0000 UTC m=+1642.114411301"
Jul 28 19:29:54.183651 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:54.183610    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn" podStartSLOduration=1.183598256 podStartE2EDuration="1.183598256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:29:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:29:54.182137523 +0000 UTC m=+1642.128622413" watchObservedRunningTime="2026-07-28 19:29:54.183598256 +0000 UTC m=+1642.130083123"
Jul 28 19:29:56.168299 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:56.168265    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z" event={"ID":"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b","Type":"ContainerStarted","Data":"cbfa0fa6376f769e879d4689f0c0c9b05d779adf96efb1852b8f22f0be885945"}
Jul 28 19:29:56.168719 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:56.168379    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"
Jul 28 19:29:56.185074 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:29:56.185024    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z" podStartSLOduration=1.819449975 podStartE2EDuration="4.185007012s" totalImagesPullingTime="2.365557037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:29:56.183286204 +0000 UTC m=+1644.129771072" watchObservedRunningTime="2026-07-28 19:29:56.185007012 +0000 UTC m=+1644.131491902"
Jul 28 19:30:02.179686 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:02.179655    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"
Jul 28 19:30:05.163584 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:05.163554    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"
Jul 28 19:30:05.164003 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:05.163815    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"
Jul 28 19:30:33.165079 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.165046    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"]
Jul 28 19:30:33.167365 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.165232    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z" podUID="eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b" containerName="maas-api" containerID="cri-o://cbfa0fa6376f769e879d4689f0c0c9b05d779adf96efb1852b8f22f0be885945" gracePeriod=30
Jul 28 19:30:33.219757 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.219726    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"]
Jul 28 19:30:33.220008 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.219964    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8" podUID="029a8673-2c24-4f12-b98b-2f7f3a9a50c1" containerName="payload-processing" containerID="cri-o://9c97b6fcfb8907d952643123842485df8e6e078b56fcc8f08887becd9a7fc75a" gracePeriod=30
Jul 28 19:30:33.226860 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.226837    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"]
Jul 28 19:30:33.227069 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.227025    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn" podUID="09a424a6-93be-4313-91fa-a193943418ac" containerName="payload-pre-processing" containerID="cri-o://bc927d76daa2dcc323a953ec06f1b420d9a46baa58921501b264b46eb6714378" gracePeriod=30
Jul 28 19:30:33.350783 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.350745    2561 generic.go:356] "Generic (PLEG): container finished" podID="09a424a6-93be-4313-91fa-a193943418ac" containerID="bc927d76daa2dcc323a953ec06f1b420d9a46baa58921501b264b46eb6714378" exitCode=0
Jul 28 19:30:33.350958 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.350850    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn" event={"ID":"09a424a6-93be-4313-91fa-a193943418ac","Type":"ContainerDied","Data":"bc927d76daa2dcc323a953ec06f1b420d9a46baa58921501b264b46eb6714378"}
Jul 28 19:30:33.352781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.352753    2561 generic.go:356] "Generic (PLEG): container finished" podID="029a8673-2c24-4f12-b98b-2f7f3a9a50c1" containerID="9c97b6fcfb8907d952643123842485df8e6e078b56fcc8f08887becd9a7fc75a" exitCode=0
Jul 28 19:30:33.352902 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.352833    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8" event={"ID":"029a8673-2c24-4f12-b98b-2f7f3a9a50c1","Type":"ContainerDied","Data":"9c97b6fcfb8907d952643123842485df8e6e078b56fcc8f08887becd9a7fc75a"}
Jul 28 19:30:33.354337 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.354311    2561 generic.go:356] "Generic (PLEG): container finished" podID="eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b" containerID="cbfa0fa6376f769e879d4689f0c0c9b05d779adf96efb1852b8f22f0be885945" exitCode=0
Jul 28 19:30:33.354450 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.354363    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z" event={"ID":"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b","Type":"ContainerDied","Data":"cbfa0fa6376f769e879d4689f0c0c9b05d779adf96efb1852b8f22f0be885945"}
Jul 28 19:30:33.464215 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.464075    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"
Jul 28 19:30:33.508025 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.508001    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"
Jul 28 19:30:33.540188 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.540165    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"
Jul 28 19:30:33.565850 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.565824    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/029a8673-2c24-4f12-b98b-2f7f3a9a50c1-plugins-config-volume\") pod \"029a8673-2c24-4f12-b98b-2f7f3a9a50c1\" (UID: \"029a8673-2c24-4f12-b98b-2f7f3a9a50c1\") "
Jul 28 19:30:33.566001 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.565909    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhr5j\" (UniqueName: \"kubernetes.io/projected/029a8673-2c24-4f12-b98b-2f7f3a9a50c1-kube-api-access-lhr5j\") pod \"029a8673-2c24-4f12-b98b-2f7f3a9a50c1\" (UID: \"029a8673-2c24-4f12-b98b-2f7f3a9a50c1\") "
Jul 28 19:30:33.566221 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.566202    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/029a8673-2c24-4f12-b98b-2f7f3a9a50c1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "029a8673-2c24-4f12-b98b-2f7f3a9a50c1" (UID: "029a8673-2c24-4f12-b98b-2f7f3a9a50c1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:30:33.568190 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.568160    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/029a8673-2c24-4f12-b98b-2f7f3a9a50c1-kube-api-access-lhr5j" (OuterVolumeSpecName: "kube-api-access-lhr5j") pod "029a8673-2c24-4f12-b98b-2f7f3a9a50c1" (UID: "029a8673-2c24-4f12-b98b-2f7f3a9a50c1"). InnerVolumeSpecName "kube-api-access-lhr5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:30:33.666766 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.666736    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09a424a6-93be-4313-91fa-a193943418ac-plugins-config-volume\") pod \"09a424a6-93be-4313-91fa-a193943418ac\" (UID: \"09a424a6-93be-4313-91fa-a193943418ac\") "
Jul 28 19:30:33.666887 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.666806    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-maas-api-tls\") pod \"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b\" (UID: \"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b\") "
Jul 28 19:30:33.666887 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.666860    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-openshift-service-ca\") pod \"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b\" (UID: \"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b\") "
Jul 28 19:30:33.667022 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.666885    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vldlm\" (UniqueName: \"kubernetes.io/projected/09a424a6-93be-4313-91fa-a193943418ac-kube-api-access-vldlm\") pod \"09a424a6-93be-4313-91fa-a193943418ac\" (UID: \"09a424a6-93be-4313-91fa-a193943418ac\") "
Jul 28 19:30:33.667022 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.666960    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xp2m2\" (UniqueName: \"kubernetes.io/projected/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-kube-api-access-xp2m2\") pod \"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b\" (UID: \"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b\") "
Jul 28 19:30:33.667203 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.667052    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09a424a6-93be-4313-91fa-a193943418ac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "09a424a6-93be-4313-91fa-a193943418ac" (UID: "09a424a6-93be-4313-91fa-a193943418ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:30:33.667265 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.667208    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b" (UID: "eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:30:33.667310 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.667258    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/029a8673-2c24-4f12-b98b-2f7f3a9a50c1-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:30:33.667310 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.667283    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhr5j\" (UniqueName: \"kubernetes.io/projected/029a8673-2c24-4f12-b98b-2f7f3a9a50c1-kube-api-access-lhr5j\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:30:33.669042 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.669020    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09a424a6-93be-4313-91fa-a193943418ac-kube-api-access-vldlm" (OuterVolumeSpecName: "kube-api-access-vldlm") pod "09a424a6-93be-4313-91fa-a193943418ac" (UID: "09a424a6-93be-4313-91fa-a193943418ac"). InnerVolumeSpecName "kube-api-access-vldlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:30:33.669141 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.669121    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-kube-api-access-xp2m2" (OuterVolumeSpecName: "kube-api-access-xp2m2") pod "eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b" (UID: "eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b"). InnerVolumeSpecName "kube-api-access-xp2m2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:30:33.669180 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.669136    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b" (UID: "eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:30:33.768496 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.768458    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09a424a6-93be-4313-91fa-a193943418ac-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:30:33.768641 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.768498    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-maas-api-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:30:33.768641 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.768513    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-openshift-service-ca\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:30:33.768641 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.768528    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vldlm\" (UniqueName: \"kubernetes.io/projected/09a424a6-93be-4313-91fa-a193943418ac-kube-api-access-vldlm\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:30:33.768641 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:33.768543    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xp2m2\" (UniqueName: \"kubernetes.io/projected/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b-kube-api-access-xp2m2\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:30:34.360399 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.360369    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"
Jul 28 19:30:34.360399 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.360391    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8" event={"ID":"029a8673-2c24-4f12-b98b-2f7f3a9a50c1","Type":"ContainerDied","Data":"542e1249bc794eec7d96cf482681f1a7ece5d1c18098f1bea30e1c362dcc2635"}
Jul 28 19:30:34.360911 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.360509    2561 scope.go:117] "RemoveContainer" containerID="9c97b6fcfb8907d952643123842485df8e6e078b56fcc8f08887becd9a7fc75a"
Jul 28 19:30:34.361778 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.361760    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"
Jul 28 19:30:34.361846 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.361754    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z" event={"ID":"eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b","Type":"ContainerDied","Data":"b8ca0817cab799d6726347a21a915ecfd5a81e97bda72a319e5c98bcf7d6a153"}
Jul 28 19:30:34.363308 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.363282    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn" event={"ID":"09a424a6-93be-4313-91fa-a193943418ac","Type":"ContainerDied","Data":"07d34ae703c35d1453637ce4e1153a0175763f9943c8faf0d4c5a224872002f8"}
Jul 28 19:30:34.363423 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.363345    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"
Jul 28 19:30:34.370893 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.370847    2561 scope.go:117] "RemoveContainer" containerID="cbfa0fa6376f769e879d4689f0c0c9b05d779adf96efb1852b8f22f0be885945"
Jul 28 19:30:34.378818 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.378801    2561 scope.go:117] "RemoveContainer" containerID="bc927d76daa2dcc323a953ec06f1b420d9a46baa58921501b264b46eb6714378"
Jul 28 19:30:34.386666 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.386645    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"]
Jul 28 19:30:34.389338 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.389317    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-16c9dd30-748588d576-brcd8"]
Jul 28 19:30:34.398598 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.398564    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"]
Jul 28 19:30:34.400244 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.400222    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-16c9dd30-6b9f4cd696-w4zhn"]
Jul 28 19:30:34.411000 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.410975    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"]
Jul 28 19:30:34.420941 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.420922    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-16c9dd30-6c498f9d88-8wd7z"]
Jul 28 19:30:34.603798 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.603764    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="029a8673-2c24-4f12-b98b-2f7f3a9a50c1" path="/var/lib/kubelet/pods/029a8673-2c24-4f12-b98b-2f7f3a9a50c1/volumes"
Jul 28 19:30:34.604662 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.604632    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09a424a6-93be-4313-91fa-a193943418ac" path="/var/lib/kubelet/pods/09a424a6-93be-4313-91fa-a193943418ac/volumes"
Jul 28 19:30:34.605548 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:34.605527    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b" path="/var/lib/kubelet/pods/eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b/volumes"
Jul 28 19:30:36.718426 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.718395    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"]
Jul 28 19:30:36.718931 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.718914    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09a424a6-93be-4313-91fa-a193943418ac" containerName="payload-pre-processing"
Jul 28 19:30:36.718974 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.718936    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="09a424a6-93be-4313-91fa-a193943418ac" containerName="payload-pre-processing"
Jul 28 19:30:36.718974 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.718969    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b" containerName="maas-api"
Jul 28 19:30:36.719032 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.718980    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b" containerName="maas-api"
Jul 28 19:30:36.719032 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.718992    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="029a8673-2c24-4f12-b98b-2f7f3a9a50c1" containerName="payload-processing"
Jul 28 19:30:36.719032 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.719002    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="029a8673-2c24-4f12-b98b-2f7f3a9a50c1" containerName="payload-processing"
Jul 28 19:30:36.719117 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.719078    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="029a8673-2c24-4f12-b98b-2f7f3a9a50c1" containerName="payload-processing"
Jul 28 19:30:36.719117 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.719090    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb5c0383-fd1d-4f5e-9eb3-9d6300323c2b" containerName="maas-api"
Jul 28 19:30:36.719117 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.719105    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="09a424a6-93be-4313-91fa-a193943418ac" containerName="payload-pre-processing"
Jul 28 19:30:36.724114 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.724095    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"
Jul 28 19:30:36.728010 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.727695    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:30:36.728010 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.727802    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z259w\""
Jul 28 19:30:36.728010 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.727860    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-84845342-serving-cert\""
Jul 28 19:30:36.728010 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.727917    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 19:30:36.730297 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.730272    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"]
Jul 28 19:30:36.899250 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.899221    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5tlh\" (UniqueName: \"kubernetes.io/projected/0eabe466-28c4-4253-a72a-4c37886e5de9-kube-api-access-f5tlh\") pod \"maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9\" (UID: \"0eabe466-28c4-4253-a72a-4c37886e5de9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"
Jul 28 19:30:36.899416 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.899281    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eabe466-28c4-4253-a72a-4c37886e5de9-maas-api-tls\") pod \"maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9\" (UID: \"0eabe466-28c4-4253-a72a-4c37886e5de9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"
Jul 28 19:30:36.899416 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.899313    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eabe466-28c4-4253-a72a-4c37886e5de9-openshift-service-ca\") pod \"maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9\" (UID: \"0eabe466-28c4-4253-a72a-4c37886e5de9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"
Jul 28 19:30:37.000067 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:36.999991    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eabe466-28c4-4253-a72a-4c37886e5de9-maas-api-tls\") pod \"maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9\" (UID: \"0eabe466-28c4-4253-a72a-4c37886e5de9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"
Jul 28 19:30:37.000067 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.000038    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eabe466-28c4-4253-a72a-4c37886e5de9-openshift-service-ca\") pod \"maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9\" (UID: \"0eabe466-28c4-4253-a72a-4c37886e5de9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"
Jul 28 19:30:37.000271 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.000149    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5tlh\" (UniqueName: \"kubernetes.io/projected/0eabe466-28c4-4253-a72a-4c37886e5de9-kube-api-access-f5tlh\") pod \"maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9\" (UID: \"0eabe466-28c4-4253-a72a-4c37886e5de9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"
Jul 28 19:30:37.000971 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.000949    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eabe466-28c4-4253-a72a-4c37886e5de9-openshift-service-ca\") pod \"maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9\" (UID: \"0eabe466-28c4-4253-a72a-4c37886e5de9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"
Jul 28 19:30:37.002989 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.002970    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eabe466-28c4-4253-a72a-4c37886e5de9-maas-api-tls\") pod \"maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9\" (UID: \"0eabe466-28c4-4253-a72a-4c37886e5de9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"
Jul 28 19:30:37.008115 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.008090    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5tlh\" (UniqueName: \"kubernetes.io/projected/0eabe466-28c4-4253-a72a-4c37886e5de9-kube-api-access-f5tlh\") pod \"maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9\" (UID: \"0eabe466-28c4-4253-a72a-4c37886e5de9\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"
Jul 28 19:30:37.037981 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.037956    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"
Jul 28 19:30:37.167966 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.167934    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"]
Jul 28 19:30:37.170143 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:30:37.170118    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eabe466_28c4_4253_a72a_4c37886e5de9.slice/crio-65d15dbb1ba538036d517d665be24238597ff7b959b7fef9a594d34c7203e667 WatchSource:0}: Error finding container 65d15dbb1ba538036d517d665be24238597ff7b959b7fef9a594d34c7203e667: Status 404 returned error can't find the container with id 65d15dbb1ba538036d517d665be24238597ff7b959b7fef9a594d34c7203e667
Jul 28 19:30:37.381919 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.381843    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9" event={"ID":"0eabe466-28c4-4253-a72a-4c37886e5de9","Type":"ContainerStarted","Data":"65d15dbb1ba538036d517d665be24238597ff7b959b7fef9a594d34c7203e667"}
Jul 28 19:30:37.382060 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.382003    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"]
Jul 28 19:30:37.386423 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.386408    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"
Jul 28 19:30:37.389854 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.389833    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-84845342-dockercfg-kctpp\""
Jul 28 19:30:37.389951 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.389912    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-84845342\""
Jul 28 19:30:37.397406 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.397386    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"]
Jul 28 19:30:37.421460 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.421430    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"]
Jul 28 19:30:37.424724 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.424708    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"
Jul 28 19:30:37.435977 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.435950    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"]
Jul 28 19:30:37.505774 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.505750    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65fbf06e-ae33-4a42-be45-3873ec0fa5ad-plugins-config-volume\") pod \"payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g\" (UID: \"65fbf06e-ae33-4a42-be45-3873ec0fa5ad\") " pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"
Jul 28 19:30:37.505907 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.505781    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8z2p\" (UniqueName: \"kubernetes.io/projected/65fbf06e-ae33-4a42-be45-3873ec0fa5ad-kube-api-access-r8z2p\") pod \"payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g\" (UID: \"65fbf06e-ae33-4a42-be45-3873ec0fa5ad\") " pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"
Jul 28 19:30:37.606688 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.606659    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65fbf06e-ae33-4a42-be45-3873ec0fa5ad-plugins-config-volume\") pod \"payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g\" (UID: \"65fbf06e-ae33-4a42-be45-3873ec0fa5ad\") " pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"
Jul 28 19:30:37.606688 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.606690    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8z2p\" (UniqueName: \"kubernetes.io/projected/65fbf06e-ae33-4a42-be45-3873ec0fa5ad-kube-api-access-r8z2p\") pod \"payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g\" (UID: \"65fbf06e-ae33-4a42-be45-3873ec0fa5ad\") " pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"
Jul 28 19:30:37.606869 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.606716    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65e4151c-7f71-4e30-b77c-bbee14f77a82-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z\" (UID: \"65e4151c-7f71-4e30-b77c-bbee14f77a82\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"
Jul 28 19:30:37.606869 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.606736    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2hpg\" (UniqueName: \"kubernetes.io/projected/65e4151c-7f71-4e30-b77c-bbee14f77a82-kube-api-access-w2hpg\") pod \"payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z\" (UID: \"65e4151c-7f71-4e30-b77c-bbee14f77a82\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"
Jul 28 19:30:37.607382 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.607362    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65fbf06e-ae33-4a42-be45-3873ec0fa5ad-plugins-config-volume\") pod \"payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g\" (UID: \"65fbf06e-ae33-4a42-be45-3873ec0fa5ad\") " pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"
Jul 28 19:30:37.619685 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.619657    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8z2p\" (UniqueName: \"kubernetes.io/projected/65fbf06e-ae33-4a42-be45-3873ec0fa5ad-kube-api-access-r8z2p\") pod \"payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g\" (UID: \"65fbf06e-ae33-4a42-be45-3873ec0fa5ad\") " pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"
Jul 28 19:30:37.698107 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.698087    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"
Jul 28 19:30:37.707645 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.707617    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65e4151c-7f71-4e30-b77c-bbee14f77a82-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z\" (UID: \"65e4151c-7f71-4e30-b77c-bbee14f77a82\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"
Jul 28 19:30:37.707757 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.707661    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2hpg\" (UniqueName: \"kubernetes.io/projected/65e4151c-7f71-4e30-b77c-bbee14f77a82-kube-api-access-w2hpg\") pod \"payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z\" (UID: \"65e4151c-7f71-4e30-b77c-bbee14f77a82\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"
Jul 28 19:30:37.708581 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.708557    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65e4151c-7f71-4e30-b77c-bbee14f77a82-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z\" (UID: \"65e4151c-7f71-4e30-b77c-bbee14f77a82\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"
Jul 28 19:30:37.730087 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.730064    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2hpg\" (UniqueName: \"kubernetes.io/projected/65e4151c-7f71-4e30-b77c-bbee14f77a82-kube-api-access-w2hpg\") pod \"payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z\" (UID: \"65e4151c-7f71-4e30-b77c-bbee14f77a82\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"
Jul 28 19:30:37.735810 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.735789    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"
Jul 28 19:30:37.841242 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.841215    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"]
Jul 28 19:30:37.844656 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:30:37.844625    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65fbf06e_ae33_4a42_be45_3873ec0fa5ad.slice/crio-9a7e731c97642fdac28f4de2b3219068ffde0d5adfe5047fb1442351c089846c WatchSource:0}: Error finding container 9a7e731c97642fdac28f4de2b3219068ffde0d5adfe5047fb1442351c089846c: Status 404 returned error can't find the container with id 9a7e731c97642fdac28f4de2b3219068ffde0d5adfe5047fb1442351c089846c
Jul 28 19:30:37.884995 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:37.884974    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"]
Jul 28 19:30:37.886364 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:30:37.886330    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e4151c_7f71_4e30_b77c_bbee14f77a82.slice/crio-0bd8220e6c3c868f8b12a4cf38c006773d72fd923ff2e2800200a76c04b52db8 WatchSource:0}: Error finding container 0bd8220e6c3c868f8b12a4cf38c006773d72fd923ff2e2800200a76c04b52db8: Status 404 returned error can't find the container with id 0bd8220e6c3c868f8b12a4cf38c006773d72fd923ff2e2800200a76c04b52db8
Jul 28 19:30:38.388613 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:38.388579    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z" event={"ID":"65e4151c-7f71-4e30-b77c-bbee14f77a82","Type":"ContainerStarted","Data":"74d1697504ee04119e23c0558e29c391ab67fb30eab2284f0b9a46c627817cfd"}
Jul 28 19:30:38.388789 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:38.388631    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z" event={"ID":"65e4151c-7f71-4e30-b77c-bbee14f77a82","Type":"ContainerStarted","Data":"0bd8220e6c3c868f8b12a4cf38c006773d72fd923ff2e2800200a76c04b52db8"}
Jul 28 19:30:38.388789 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:38.388689    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"
Jul 28 19:30:38.390244 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:38.390217    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g" event={"ID":"65fbf06e-ae33-4a42-be45-3873ec0fa5ad","Type":"ContainerStarted","Data":"b0d9d05dbd4797969b930d5d547e221ed28f189c94d6fa312ce95c8c71b73ffb"}
Jul 28 19:30:38.390369 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:38.390261    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"
Jul 28 19:30:38.390369 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:38.390281    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g" event={"ID":"65fbf06e-ae33-4a42-be45-3873ec0fa5ad","Type":"ContainerStarted","Data":"9a7e731c97642fdac28f4de2b3219068ffde0d5adfe5047fb1442351c089846c"}
Jul 28 19:30:38.391620 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:38.391598    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9" event={"ID":"0eabe466-28c4-4253-a72a-4c37886e5de9","Type":"ContainerStarted","Data":"05a830da51201bf7e34dae909d1371c9130013975005a5af4104f5d52c69e5f3"}
Jul 28 19:30:38.391746 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:38.391731    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"
Jul 28 19:30:38.406472 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:38.406416    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z" podStartSLOduration=1.406396883 podStartE2EDuration="1.406396883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:30:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:30:38.403706291 +0000 UTC m=+1686.350191181" watchObservedRunningTime="2026-07-28 19:30:38.406396883 +0000 UTC m=+1686.352881751"
Jul 28 19:30:38.421386 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:38.421332    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g" podStartSLOduration=1.421317016 podStartE2EDuration="1.421317016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:30:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:30:38.419427263 +0000 UTC m=+1686.365912156" watchObservedRunningTime="2026-07-28 19:30:38.421317016 +0000 UTC m=+1686.367801884"
Jul 28 19:30:38.438195 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:38.438142    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9" podStartSLOduration=1.950692517 podStartE2EDuration="2.438126058s" totalImagesPullingTime="487.433541ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:30:38.43685411 +0000 UTC m=+1686.383338992" watchObservedRunningTime="2026-07-28 19:30:38.438126058 +0000 UTC m=+1686.384610925"
Jul 28 19:30:44.402322 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:44.402293    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"
Jul 28 19:30:49.398219 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:49.398182    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"
Jul 28 19:30:49.398812 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:30:49.398254    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"
Jul 28 19:31:08.234110 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.234073    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"]
Jul 28 19:31:08.234970 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.234897    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9" podUID="0eabe466-28c4-4253-a72a-4c37886e5de9" containerName="maas-api" containerID="cri-o://05a830da51201bf7e34dae909d1371c9130013975005a5af4104f5d52c69e5f3" gracePeriod=30
Jul 28 19:31:08.296110 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.296088    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"]
Jul 28 19:31:08.296311 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.296287    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g" podUID="65fbf06e-ae33-4a42-be45-3873ec0fa5ad" containerName="payload-processing" containerID="cri-o://b0d9d05dbd4797969b930d5d547e221ed28f189c94d6fa312ce95c8c71b73ffb" gracePeriod=30
Jul 28 19:31:08.305007 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.304979    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"]
Jul 28 19:31:08.305412 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.305234    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z" podUID="65e4151c-7f71-4e30-b77c-bbee14f77a82" containerName="payload-pre-processing" containerID="cri-o://74d1697504ee04119e23c0558e29c391ab67fb30eab2284f0b9a46c627817cfd" gracePeriod=30
Jul 28 19:31:08.538448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.538413    2561 generic.go:356] "Generic (PLEG): container finished" podID="0eabe466-28c4-4253-a72a-4c37886e5de9" containerID="05a830da51201bf7e34dae909d1371c9130013975005a5af4104f5d52c69e5f3" exitCode=0
Jul 28 19:31:08.538701 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.538671    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9" event={"ID":"0eabe466-28c4-4253-a72a-4c37886e5de9","Type":"ContainerDied","Data":"05a830da51201bf7e34dae909d1371c9130013975005a5af4104f5d52c69e5f3"}
Jul 28 19:31:08.540850 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.540799    2561 generic.go:356] "Generic (PLEG): container finished" podID="65e4151c-7f71-4e30-b77c-bbee14f77a82" containerID="74d1697504ee04119e23c0558e29c391ab67fb30eab2284f0b9a46c627817cfd" exitCode=0
Jul 28 19:31:08.540971 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.540838    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z" event={"ID":"65e4151c-7f71-4e30-b77c-bbee14f77a82","Type":"ContainerDied","Data":"74d1697504ee04119e23c0558e29c391ab67fb30eab2284f0b9a46c627817cfd"}
Jul 28 19:31:08.544012 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.543990    2561 generic.go:356] "Generic (PLEG): container finished" podID="65fbf06e-ae33-4a42-be45-3873ec0fa5ad" containerID="b0d9d05dbd4797969b930d5d547e221ed28f189c94d6fa312ce95c8c71b73ffb" exitCode=0
Jul 28 19:31:08.544116 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.544046    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g" event={"ID":"65fbf06e-ae33-4a42-be45-3873ec0fa5ad","Type":"ContainerDied","Data":"b0d9d05dbd4797969b930d5d547e221ed28f189c94d6fa312ce95c8c71b73ffb"}
Jul 28 19:31:08.576311 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.576290    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"
Jul 28 19:31:08.607230 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.607206    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"
Jul 28 19:31:08.609389 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.609369    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"
Jul 28 19:31:08.654102 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.654079    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8z2p\" (UniqueName: \"kubernetes.io/projected/65fbf06e-ae33-4a42-be45-3873ec0fa5ad-kube-api-access-r8z2p\") pod \"65fbf06e-ae33-4a42-be45-3873ec0fa5ad\" (UID: \"65fbf06e-ae33-4a42-be45-3873ec0fa5ad\") "
Jul 28 19:31:08.654245 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.654154    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65fbf06e-ae33-4a42-be45-3873ec0fa5ad-plugins-config-volume\") pod \"65fbf06e-ae33-4a42-be45-3873ec0fa5ad\" (UID: \"65fbf06e-ae33-4a42-be45-3873ec0fa5ad\") "
Jul 28 19:31:08.654627 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.654595    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65fbf06e-ae33-4a42-be45-3873ec0fa5ad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "65fbf06e-ae33-4a42-be45-3873ec0fa5ad" (UID: "65fbf06e-ae33-4a42-be45-3873ec0fa5ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:31:08.656219 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.656197    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65fbf06e-ae33-4a42-be45-3873ec0fa5ad-kube-api-access-r8z2p" (OuterVolumeSpecName: "kube-api-access-r8z2p") pod "65fbf06e-ae33-4a42-be45-3873ec0fa5ad" (UID: "65fbf06e-ae33-4a42-be45-3873ec0fa5ad"). InnerVolumeSpecName "kube-api-access-r8z2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:31:08.755115 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.755059    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eabe466-28c4-4253-a72a-4c37886e5de9-maas-api-tls\") pod \"0eabe466-28c4-4253-a72a-4c37886e5de9\" (UID: \"0eabe466-28c4-4253-a72a-4c37886e5de9\") "
Jul 28 19:31:08.755204 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.755150    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65e4151c-7f71-4e30-b77c-bbee14f77a82-plugins-config-volume\") pod \"65e4151c-7f71-4e30-b77c-bbee14f77a82\" (UID: \"65e4151c-7f71-4e30-b77c-bbee14f77a82\") "
Jul 28 19:31:08.755204 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.755189    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eabe466-28c4-4253-a72a-4c37886e5de9-openshift-service-ca\") pod \"0eabe466-28c4-4253-a72a-4c37886e5de9\" (UID: \"0eabe466-28c4-4253-a72a-4c37886e5de9\") "
Jul 28 19:31:08.755288 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.755206    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2hpg\" (UniqueName: \"kubernetes.io/projected/65e4151c-7f71-4e30-b77c-bbee14f77a82-kube-api-access-w2hpg\") pod \"65e4151c-7f71-4e30-b77c-bbee14f77a82\" (UID: \"65e4151c-7f71-4e30-b77c-bbee14f77a82\") "
Jul 28 19:31:08.755288 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.755239    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5tlh\" (UniqueName: \"kubernetes.io/projected/0eabe466-28c4-4253-a72a-4c37886e5de9-kube-api-access-f5tlh\") pod \"0eabe466-28c4-4253-a72a-4c37886e5de9\" (UID: \"0eabe466-28c4-4253-a72a-4c37886e5de9\") "
Jul 28 19:31:08.755425 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.755407    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8z2p\" (UniqueName: \"kubernetes.io/projected/65fbf06e-ae33-4a42-be45-3873ec0fa5ad-kube-api-access-r8z2p\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:31:08.755512 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.755433    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65fbf06e-ae33-4a42-be45-3873ec0fa5ad-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:31:08.755585 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.755546    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eabe466-28c4-4253-a72a-4c37886e5de9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0eabe466-28c4-4253-a72a-4c37886e5de9" (UID: "0eabe466-28c4-4253-a72a-4c37886e5de9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:31:08.755645 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.755584    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65e4151c-7f71-4e30-b77c-bbee14f77a82-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "65e4151c-7f71-4e30-b77c-bbee14f77a82" (UID: "65e4151c-7f71-4e30-b77c-bbee14f77a82"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:31:08.757106 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.757081    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0eabe466-28c4-4253-a72a-4c37886e5de9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0eabe466-28c4-4253-a72a-4c37886e5de9" (UID: "0eabe466-28c4-4253-a72a-4c37886e5de9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:31:08.757266 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.757239    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65e4151c-7f71-4e30-b77c-bbee14f77a82-kube-api-access-w2hpg" (OuterVolumeSpecName: "kube-api-access-w2hpg") pod "65e4151c-7f71-4e30-b77c-bbee14f77a82" (UID: "65e4151c-7f71-4e30-b77c-bbee14f77a82"). InnerVolumeSpecName "kube-api-access-w2hpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:31:08.757321 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.757263    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0eabe466-28c4-4253-a72a-4c37886e5de9-kube-api-access-f5tlh" (OuterVolumeSpecName: "kube-api-access-f5tlh") pod "0eabe466-28c4-4253-a72a-4c37886e5de9" (UID: "0eabe466-28c4-4253-a72a-4c37886e5de9"). InnerVolumeSpecName "kube-api-access-f5tlh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:31:08.856094 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.856071    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65e4151c-7f71-4e30-b77c-bbee14f77a82-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:31:08.856094 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.856092    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eabe466-28c4-4253-a72a-4c37886e5de9-openshift-service-ca\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:31:08.856229 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.856101    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2hpg\" (UniqueName: \"kubernetes.io/projected/65e4151c-7f71-4e30-b77c-bbee14f77a82-kube-api-access-w2hpg\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:31:08.856229 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.856110    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5tlh\" (UniqueName: \"kubernetes.io/projected/0eabe466-28c4-4253-a72a-4c37886e5de9-kube-api-access-f5tlh\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:31:08.856229 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:08.856119    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eabe466-28c4-4253-a72a-4c37886e5de9-maas-api-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:31:09.549587 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:09.549507    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"
Jul 28 19:31:09.549587 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:09.549526    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9" event={"ID":"0eabe466-28c4-4253-a72a-4c37886e5de9","Type":"ContainerDied","Data":"65d15dbb1ba538036d517d665be24238597ff7b959b7fef9a594d34c7203e667"}
Jul 28 19:31:09.550086 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:09.549598    2561 scope.go:117] "RemoveContainer" containerID="05a830da51201bf7e34dae909d1371c9130013975005a5af4104f5d52c69e5f3"
Jul 28 19:31:09.550965 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:09.550947    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"
Jul 28 19:31:09.551041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:09.550974    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z" event={"ID":"65e4151c-7f71-4e30-b77c-bbee14f77a82","Type":"ContainerDied","Data":"0bd8220e6c3c868f8b12a4cf38c006773d72fd923ff2e2800200a76c04b52db8"}
Jul 28 19:31:09.552552 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:09.552529    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g" event={"ID":"65fbf06e-ae33-4a42-be45-3873ec0fa5ad","Type":"ContainerDied","Data":"9a7e731c97642fdac28f4de2b3219068ffde0d5adfe5047fb1442351c089846c"}
Jul 28 19:31:09.552791 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:09.552558    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"
Jul 28 19:31:09.559611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:09.559549    2561 scope.go:117] "RemoveContainer" containerID="74d1697504ee04119e23c0558e29c391ab67fb30eab2284f0b9a46c627817cfd"
Jul 28 19:31:09.571548 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:09.571530    2561 scope.go:117] "RemoveContainer" containerID="b0d9d05dbd4797969b930d5d547e221ed28f189c94d6fa312ce95c8c71b73ffb"
Jul 28 19:31:09.575567 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:09.575544    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"]
Jul 28 19:31:09.578856 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:09.578837    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-84845342-5c898cd54f-p9xp9"]
Jul 28 19:31:09.589426 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:09.589404    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"]
Jul 28 19:31:09.596145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:09.596123    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-84845342-ffcfb696d-tfz9z"]
Jul 28 19:31:09.609098 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:09.609075    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"]
Jul 28 19:31:09.613613 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:09.613595    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-84845342-5764975b8f-2bv6g"]
Jul 28 19:31:10.603101 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:10.603065    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0eabe466-28c4-4253-a72a-4c37886e5de9" path="/var/lib/kubelet/pods/0eabe466-28c4-4253-a72a-4c37886e5de9/volumes"
Jul 28 19:31:10.603462 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:10.603442    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65e4151c-7f71-4e30-b77c-bbee14f77a82" path="/var/lib/kubelet/pods/65e4151c-7f71-4e30-b77c-bbee14f77a82/volumes"
Jul 28 19:31:10.603822 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:10.603808    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65fbf06e-ae33-4a42-be45-3873ec0fa5ad" path="/var/lib/kubelet/pods/65fbf06e-ae33-4a42-be45-3873ec0fa5ad/volumes"
Jul 28 19:31:11.796713 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.796679    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"]
Jul 28 19:31:11.797277 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.797257    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0eabe466-28c4-4253-a72a-4c37886e5de9" containerName="maas-api"
Jul 28 19:31:11.797321 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.797282    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0eabe466-28c4-4253-a72a-4c37886e5de9" containerName="maas-api"
Jul 28 19:31:11.797321 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.797314    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65e4151c-7f71-4e30-b77c-bbee14f77a82" containerName="payload-pre-processing"
Jul 28 19:31:11.797393 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.797323    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="65e4151c-7f71-4e30-b77c-bbee14f77a82" containerName="payload-pre-processing"
Jul 28 19:31:11.797393 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.797352    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65fbf06e-ae33-4a42-be45-3873ec0fa5ad" containerName="payload-processing"
Jul 28 19:31:11.797393 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.797361    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="65fbf06e-ae33-4a42-be45-3873ec0fa5ad" containerName="payload-processing"
Jul 28 19:31:11.797497 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.797445    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0eabe466-28c4-4253-a72a-4c37886e5de9" containerName="maas-api"
Jul 28 19:31:11.797497 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.797462    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="65e4151c-7f71-4e30-b77c-bbee14f77a82" containerName="payload-pre-processing"
Jul 28 19:31:11.797497 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.797474    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="65fbf06e-ae33-4a42-be45-3873ec0fa5ad" containerName="payload-processing"
Jul 28 19:31:11.802946 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.802925    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"
Jul 28 19:31:11.805285 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.805265    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-43235862-serving-cert\""
Jul 28 19:31:11.805498 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.805450    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:31:11.806409 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.806389    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z259w\""
Jul 28 19:31:11.806528 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.806456    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 19:31:11.809245 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.809219    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"]
Jul 28 19:31:11.880204 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.880184    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-openshift-service-ca\") pod \"maas-api-e2e-ait-43235862-5bb589f88c-2zvks\" (UID: \"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"
Jul 28 19:31:11.880299 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.880223    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkfz6\" (UniqueName: \"kubernetes.io/projected/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-kube-api-access-lkfz6\") pod \"maas-api-e2e-ait-43235862-5bb589f88c-2zvks\" (UID: \"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"
Jul 28 19:31:11.880342 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.880311    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-maas-api-tls\") pod \"maas-api-e2e-ait-43235862-5bb589f88c-2zvks\" (UID: \"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"
Jul 28 19:31:11.981682 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.981657    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-openshift-service-ca\") pod \"maas-api-e2e-ait-43235862-5bb589f88c-2zvks\" (UID: \"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"
Jul 28 19:31:11.981776 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.981693    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkfz6\" (UniqueName: \"kubernetes.io/projected/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-kube-api-access-lkfz6\") pod \"maas-api-e2e-ait-43235862-5bb589f88c-2zvks\" (UID: \"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"
Jul 28 19:31:11.981776 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.981725    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-maas-api-tls\") pod \"maas-api-e2e-ait-43235862-5bb589f88c-2zvks\" (UID: \"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"
Jul 28 19:31:11.982261 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.982239    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-openshift-service-ca\") pod \"maas-api-e2e-ait-43235862-5bb589f88c-2zvks\" (UID: \"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"
Jul 28 19:31:11.984007 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.983983    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-maas-api-tls\") pod \"maas-api-e2e-ait-43235862-5bb589f88c-2zvks\" (UID: \"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"
Jul 28 19:31:11.989817 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:11.989792    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkfz6\" (UniqueName: \"kubernetes.io/projected/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-kube-api-access-lkfz6\") pod \"maas-api-e2e-ait-43235862-5bb589f88c-2zvks\" (UID: \"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"
Jul 28 19:31:12.114086 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.114033    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"
Jul 28 19:31:12.241027 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.240975    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"]
Jul 28 19:31:12.243254 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:31:12.243223    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bd95977_49ef_4ea9_bd3a_1d47abfcf9d2.slice/crio-ad273d48d158c05b24d5506e94b11867ca7f13712bab3b14dadf7f67c23f380c WatchSource:0}: Error finding container ad273d48d158c05b24d5506e94b11867ca7f13712bab3b14dadf7f67c23f380c: Status 404 returned error can't find the container with id ad273d48d158c05b24d5506e94b11867ca7f13712bab3b14dadf7f67c23f380c
Jul 28 19:31:12.459990 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.459959    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"]
Jul 28 19:31:12.464989 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.464969    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"
Jul 28 19:31:12.467588 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.467570    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-43235862\""
Jul 28 19:31:12.468418 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.468401    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-43235862-dockercfg-fgt8s\""
Jul 28 19:31:12.474041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.474023    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"]
Jul 28 19:31:12.482082 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.482064    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"]
Jul 28 19:31:12.485504 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.485455    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"
Jul 28 19:31:12.494861 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.494840    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"]
Jul 28 19:31:12.570410 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.570380    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks" event={"ID":"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2","Type":"ContainerStarted","Data":"ad273d48d158c05b24d5506e94b11867ca7f13712bab3b14dadf7f67c23f380c"}
Jul 28 19:31:12.586832 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.586806    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3077dbb6-c04e-4b59-b66f-88613e148c37-plugins-config-volume\") pod \"payload-processing-e2e-ait-43235862-548c589d9d-5l6cd\" (UID: \"3077dbb6-c04e-4b59-b66f-88613e148c37\") " pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"
Jul 28 19:31:12.586950 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.586851    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp2mv\" (UniqueName: \"kubernetes.io/projected/3077dbb6-c04e-4b59-b66f-88613e148c37-kube-api-access-sp2mv\") pod \"payload-processing-e2e-ait-43235862-548c589d9d-5l6cd\" (UID: \"3077dbb6-c04e-4b59-b66f-88613e148c37\") " pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"
Jul 28 19:31:12.586950 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.586904    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaa35347-b733-44aa-a2be-a3c7227a822b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn\" (UID: \"aaa35347-b733-44aa-a2be-a3c7227a822b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"
Jul 28 19:31:12.586950 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.586936    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk2p2\" (UniqueName: \"kubernetes.io/projected/aaa35347-b733-44aa-a2be-a3c7227a822b-kube-api-access-qk2p2\") pod \"payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn\" (UID: \"aaa35347-b733-44aa-a2be-a3c7227a822b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"
Jul 28 19:31:12.687747 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.687719    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3077dbb6-c04e-4b59-b66f-88613e148c37-plugins-config-volume\") pod \"payload-processing-e2e-ait-43235862-548c589d9d-5l6cd\" (UID: \"3077dbb6-c04e-4b59-b66f-88613e148c37\") " pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"
Jul 28 19:31:12.687879 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.687758    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp2mv\" (UniqueName: \"kubernetes.io/projected/3077dbb6-c04e-4b59-b66f-88613e148c37-kube-api-access-sp2mv\") pod \"payload-processing-e2e-ait-43235862-548c589d9d-5l6cd\" (UID: \"3077dbb6-c04e-4b59-b66f-88613e148c37\") " pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"
Jul 28 19:31:12.687921 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.687884    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaa35347-b733-44aa-a2be-a3c7227a822b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn\" (UID: \"aaa35347-b733-44aa-a2be-a3c7227a822b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"
Jul 28 19:31:12.687921 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.687912    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk2p2\" (UniqueName: \"kubernetes.io/projected/aaa35347-b733-44aa-a2be-a3c7227a822b-kube-api-access-qk2p2\") pod \"payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn\" (UID: \"aaa35347-b733-44aa-a2be-a3c7227a822b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"
Jul 28 19:31:12.688501 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.688465    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3077dbb6-c04e-4b59-b66f-88613e148c37-plugins-config-volume\") pod \"payload-processing-e2e-ait-43235862-548c589d9d-5l6cd\" (UID: \"3077dbb6-c04e-4b59-b66f-88613e148c37\") " pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"
Jul 28 19:31:12.688697 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.688678    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaa35347-b733-44aa-a2be-a3c7227a822b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn\" (UID: \"aaa35347-b733-44aa-a2be-a3c7227a822b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"
Jul 28 19:31:12.701537 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.701513    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk2p2\" (UniqueName: \"kubernetes.io/projected/aaa35347-b733-44aa-a2be-a3c7227a822b-kube-api-access-qk2p2\") pod \"payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn\" (UID: \"aaa35347-b733-44aa-a2be-a3c7227a822b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"
Jul 28 19:31:12.701705 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.701687    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp2mv\" (UniqueName: \"kubernetes.io/projected/3077dbb6-c04e-4b59-b66f-88613e148c37-kube-api-access-sp2mv\") pod \"payload-processing-e2e-ait-43235862-548c589d9d-5l6cd\" (UID: \"3077dbb6-c04e-4b59-b66f-88613e148c37\") " pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"
Jul 28 19:31:12.775615 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.775561    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"
Jul 28 19:31:12.797366 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.797348    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"
Jul 28 19:31:12.924631 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.924529    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"]
Jul 28 19:31:12.927855 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:31:12.927826    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa35347_b733_44aa_a2be_a3c7227a822b.slice/crio-badf8af630b10ace30b706bd64719f4a71b20d04b53afbefcd925329ba2ae6c4 WatchSource:0}: Error finding container badf8af630b10ace30b706bd64719f4a71b20d04b53afbefcd925329ba2ae6c4: Status 404 returned error can't find the container with id badf8af630b10ace30b706bd64719f4a71b20d04b53afbefcd925329ba2ae6c4
Jul 28 19:31:12.950613 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:12.950588    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"]
Jul 28 19:31:12.956553 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:31:12.956527    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3077dbb6_c04e_4b59_b66f_88613e148c37.slice/crio-12bc78a78a14e87e510034b73c3d9d9da1f40e3227bf8ebc20c4af995c412f57 WatchSource:0}: Error finding container 12bc78a78a14e87e510034b73c3d9d9da1f40e3227bf8ebc20c4af995c412f57: Status 404 returned error can't find the container with id 12bc78a78a14e87e510034b73c3d9d9da1f40e3227bf8ebc20c4af995c412f57
Jul 28 19:31:13.576939 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:13.576904    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks" event={"ID":"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2","Type":"ContainerStarted","Data":"bd0e406185b24f28e50a376dd09564f79917b2a9a1724138d2e4036a1d438c41"}
Jul 28 19:31:13.577131 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:13.576960    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"
Jul 28 19:31:13.578337 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:13.578313    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd" event={"ID":"3077dbb6-c04e-4b59-b66f-88613e148c37","Type":"ContainerStarted","Data":"ada89fb0ed280f293ca59179f583dcd7c0f32f077cd41f7d7651c8148fd010ee"}
Jul 28 19:31:13.578446 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:13.578364    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"
Jul 28 19:31:13.578446 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:13.578392    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd" event={"ID":"3077dbb6-c04e-4b59-b66f-88613e148c37","Type":"ContainerStarted","Data":"12bc78a78a14e87e510034b73c3d9d9da1f40e3227bf8ebc20c4af995c412f57"}
Jul 28 19:31:13.579731 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:13.579713    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn" event={"ID":"aaa35347-b733-44aa-a2be-a3c7227a822b","Type":"ContainerStarted","Data":"f10bd41be6f42c6962f011932f4420755f5d3842d38d3faa9061b0cebe3ec993"}
Jul 28 19:31:13.579815 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:13.579747    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn" event={"ID":"aaa35347-b733-44aa-a2be-a3c7227a822b","Type":"ContainerStarted","Data":"badf8af630b10ace30b706bd64719f4a71b20d04b53afbefcd925329ba2ae6c4"}
Jul 28 19:31:13.579815 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:13.579775    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"
Jul 28 19:31:13.607789 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:13.607736    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks" podStartSLOduration=1.996980279 podStartE2EDuration="2.607719426s" totalImagesPullingTime="610.739147ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:31:13.60657485 +0000 UTC m=+1721.553059742" watchObservedRunningTime="2026-07-28 19:31:13.607719426 +0000 UTC m=+1721.554204294"
Jul 28 19:31:13.630449 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:13.630409    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn" podStartSLOduration=1.6303958060000001 podStartE2EDuration="1.630395806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:31:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:31:13.628736075 +0000 UTC m=+1721.575220968" watchObservedRunningTime="2026-07-28 19:31:13.630395806 +0000 UTC m=+1721.576880686"
Jul 28 19:31:13.646759 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:13.646714    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd" podStartSLOduration=1.6466973569999999 podStartE2EDuration="1.646697357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:31:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:31:13.644590254 +0000 UTC m=+1721.591075153" watchObservedRunningTime="2026-07-28 19:31:13.646697357 +0000 UTC m=+1721.593182228"
Jul 28 19:31:19.589298 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:19.589271    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"
Jul 28 19:31:24.587330 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:24.587296    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"
Jul 28 19:31:24.587880 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:24.587374    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"
Jul 28 19:31:42.521499 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.521423    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"]
Jul 28 19:31:42.522020 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.521717    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks" podUID="9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2" containerName="maas-api" containerID="cri-o://bd0e406185b24f28e50a376dd09564f79917b2a9a1724138d2e4036a1d438c41" gracePeriod=30
Jul 28 19:31:42.571668 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.571637    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"]
Jul 28 19:31:42.571900 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.571848    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd" podUID="3077dbb6-c04e-4b59-b66f-88613e148c37" containerName="payload-processing" containerID="cri-o://ada89fb0ed280f293ca59179f583dcd7c0f32f077cd41f7d7651c8148fd010ee" gracePeriod=30
Jul 28 19:31:42.576316 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.576228    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"]
Jul 28 19:31:42.576476 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.576446    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn" podUID="aaa35347-b733-44aa-a2be-a3c7227a822b" containerName="payload-pre-processing" containerID="cri-o://f10bd41be6f42c6962f011932f4420755f5d3842d38d3faa9061b0cebe3ec993" gracePeriod=30
Jul 28 19:31:42.729264 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.729221    2561 generic.go:356] "Generic (PLEG): container finished" podID="9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2" containerID="bd0e406185b24f28e50a376dd09564f79917b2a9a1724138d2e4036a1d438c41" exitCode=0
Jul 28 19:31:42.729414 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.729356    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks" event={"ID":"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2","Type":"ContainerDied","Data":"bd0e406185b24f28e50a376dd09564f79917b2a9a1724138d2e4036a1d438c41"}
Jul 28 19:31:42.730970 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.730839    2561 generic.go:356] "Generic (PLEG): container finished" podID="3077dbb6-c04e-4b59-b66f-88613e148c37" containerID="ada89fb0ed280f293ca59179f583dcd7c0f32f077cd41f7d7651c8148fd010ee" exitCode=0
Jul 28 19:31:42.730970 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.730918    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd" event={"ID":"3077dbb6-c04e-4b59-b66f-88613e148c37","Type":"ContainerDied","Data":"ada89fb0ed280f293ca59179f583dcd7c0f32f077cd41f7d7651c8148fd010ee"}
Jul 28 19:31:42.732719 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.732681    2561 generic.go:356] "Generic (PLEG): container finished" podID="aaa35347-b733-44aa-a2be-a3c7227a822b" containerID="f10bd41be6f42c6962f011932f4420755f5d3842d38d3faa9061b0cebe3ec993" exitCode=0
Jul 28 19:31:42.733008 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.732766    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn" event={"ID":"aaa35347-b733-44aa-a2be-a3c7227a822b","Type":"ContainerDied","Data":"f10bd41be6f42c6962f011932f4420755f5d3842d38d3faa9061b0cebe3ec993"}
Jul 28 19:31:42.830460 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.830433    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"
Jul 28 19:31:42.856788 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.856766    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"
Jul 28 19:31:42.867182 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.867107    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"
Jul 28 19:31:42.931871 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.931848    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3077dbb6-c04e-4b59-b66f-88613e148c37-plugins-config-volume\") pod \"3077dbb6-c04e-4b59-b66f-88613e148c37\" (UID: \"3077dbb6-c04e-4b59-b66f-88613e148c37\") "
Jul 28 19:31:42.932020 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.931889    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sp2mv\" (UniqueName: \"kubernetes.io/projected/3077dbb6-c04e-4b59-b66f-88613e148c37-kube-api-access-sp2mv\") pod \"3077dbb6-c04e-4b59-b66f-88613e148c37\" (UID: \"3077dbb6-c04e-4b59-b66f-88613e148c37\") "
Jul 28 19:31:42.932020 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.931920    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-maas-api-tls\") pod \"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2\" (UID: \"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2\") "
Jul 28 19:31:42.932020 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.931951    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qk2p2\" (UniqueName: \"kubernetes.io/projected/aaa35347-b733-44aa-a2be-a3c7227a822b-kube-api-access-qk2p2\") pod \"aaa35347-b733-44aa-a2be-a3c7227a822b\" (UID: \"aaa35347-b733-44aa-a2be-a3c7227a822b\") "
Jul 28 19:31:42.932020 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.931989    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-openshift-service-ca\") pod \"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2\" (UID: \"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2\") "
Jul 28 19:31:42.932020 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.932015    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkfz6\" (UniqueName: \"kubernetes.io/projected/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-kube-api-access-lkfz6\") pod \"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2\" (UID: \"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2\") "
Jul 28 19:31:42.932300 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.932155    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaa35347-b733-44aa-a2be-a3c7227a822b-plugins-config-volume\") pod \"aaa35347-b733-44aa-a2be-a3c7227a822b\" (UID: \"aaa35347-b733-44aa-a2be-a3c7227a822b\") "
Jul 28 19:31:42.932300 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.932272    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3077dbb6-c04e-4b59-b66f-88613e148c37-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3077dbb6-c04e-4b59-b66f-88613e148c37" (UID: "3077dbb6-c04e-4b59-b66f-88613e148c37"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:31:42.932512 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.932369    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2" (UID: "9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:31:42.932645 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.932622    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-openshift-service-ca\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:31:42.932724 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.932649    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3077dbb6-c04e-4b59-b66f-88613e148c37-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:31:42.932724 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.932688    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaa35347-b733-44aa-a2be-a3c7227a822b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aaa35347-b733-44aa-a2be-a3c7227a822b" (UID: "aaa35347-b733-44aa-a2be-a3c7227a822b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:31:42.934134 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.934113    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2" (UID: "9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:31:42.934222 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.934173    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaa35347-b733-44aa-a2be-a3c7227a822b-kube-api-access-qk2p2" (OuterVolumeSpecName: "kube-api-access-qk2p2") pod "aaa35347-b733-44aa-a2be-a3c7227a822b" (UID: "aaa35347-b733-44aa-a2be-a3c7227a822b"). InnerVolumeSpecName "kube-api-access-qk2p2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:31:42.934428 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.934405    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3077dbb6-c04e-4b59-b66f-88613e148c37-kube-api-access-sp2mv" (OuterVolumeSpecName: "kube-api-access-sp2mv") pod "3077dbb6-c04e-4b59-b66f-88613e148c37" (UID: "3077dbb6-c04e-4b59-b66f-88613e148c37"). InnerVolumeSpecName "kube-api-access-sp2mv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:31:42.934428 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:42.934412    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-kube-api-access-lkfz6" (OuterVolumeSpecName: "kube-api-access-lkfz6") pod "9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2" (UID: "9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2"). InnerVolumeSpecName "kube-api-access-lkfz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:31:43.033804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.033746    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkfz6\" (UniqueName: \"kubernetes.io/projected/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-kube-api-access-lkfz6\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:31:43.033804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.033774    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaa35347-b733-44aa-a2be-a3c7227a822b-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:31:43.033804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.033783    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sp2mv\" (UniqueName: \"kubernetes.io/projected/3077dbb6-c04e-4b59-b66f-88613e148c37-kube-api-access-sp2mv\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:31:43.033804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.033792    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2-maas-api-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:31:43.033804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.033800    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qk2p2\" (UniqueName: \"kubernetes.io/projected/aaa35347-b733-44aa-a2be-a3c7227a822b-kube-api-access-qk2p2\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:31:43.738982 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.738952    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"
Jul 28 19:31:43.739408 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.738999    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks" event={"ID":"9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2","Type":"ContainerDied","Data":"ad273d48d158c05b24d5506e94b11867ca7f13712bab3b14dadf7f67c23f380c"}
Jul 28 19:31:43.739408 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.739068    2561 scope.go:117] "RemoveContainer" containerID="bd0e406185b24f28e50a376dd09564f79917b2a9a1724138d2e4036a1d438c41"
Jul 28 19:31:43.740585 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.740562    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"
Jul 28 19:31:43.740780 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.740600    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd" event={"ID":"3077dbb6-c04e-4b59-b66f-88613e148c37","Type":"ContainerDied","Data":"12bc78a78a14e87e510034b73c3d9d9da1f40e3227bf8ebc20c4af995c412f57"}
Jul 28 19:31:43.742422 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.742384    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn" event={"ID":"aaa35347-b733-44aa-a2be-a3c7227a822b","Type":"ContainerDied","Data":"badf8af630b10ace30b706bd64719f4a71b20d04b53afbefcd925329ba2ae6c4"}
Jul 28 19:31:43.742422 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.742409    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"
Jul 28 19:31:43.749729 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.749704    2561 scope.go:117] "RemoveContainer" containerID="ada89fb0ed280f293ca59179f583dcd7c0f32f077cd41f7d7651c8148fd010ee"
Jul 28 19:31:43.759206 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.759187    2561 scope.go:117] "RemoveContainer" containerID="f10bd41be6f42c6962f011932f4420755f5d3842d38d3faa9061b0cebe3ec993"
Jul 28 19:31:43.763579 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.763556    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"]
Jul 28 19:31:43.769575 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.769553    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-43235862-5bb589f88c-2zvks"]
Jul 28 19:31:43.779865 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.779844    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"]
Jul 28 19:31:43.785503 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.785465    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-43235862-587b4d765b-gllfn"]
Jul 28 19:31:43.795585 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.795523    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"]
Jul 28 19:31:43.801756 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:43.801734    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-43235862-548c589d9d-5l6cd"]
Jul 28 19:31:44.605516 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:44.605466    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3077dbb6-c04e-4b59-b66f-88613e148c37" path="/var/lib/kubelet/pods/3077dbb6-c04e-4b59-b66f-88613e148c37/volumes"
Jul 28 19:31:44.605922 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:44.605907    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2" path="/var/lib/kubelet/pods/9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2/volumes"
Jul 28 19:31:44.606268 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:44.606253    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaa35347-b733-44aa-a2be-a3c7227a822b" path="/var/lib/kubelet/pods/aaa35347-b733-44aa-a2be-a3c7227a822b/volumes"
Jul 28 19:31:46.960508 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:46.960455    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"]
Jul 28 19:31:46.960883 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:46.960803    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3077dbb6-c04e-4b59-b66f-88613e148c37" containerName="payload-processing"
Jul 28 19:31:46.960883 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:46.960815    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3077dbb6-c04e-4b59-b66f-88613e148c37" containerName="payload-processing"
Jul 28 19:31:46.960883 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:46.960835    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2" containerName="maas-api"
Jul 28 19:31:46.960883 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:46.960842    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2" containerName="maas-api"
Jul 28 19:31:46.960883 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:46.960862    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaa35347-b733-44aa-a2be-a3c7227a822b" containerName="payload-pre-processing"
Jul 28 19:31:46.960883 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:46.960868    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaa35347-b733-44aa-a2be-a3c7227a822b" containerName="payload-pre-processing"
Jul 28 19:31:46.961075 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:46.960920    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3077dbb6-c04e-4b59-b66f-88613e148c37" containerName="payload-processing"
Jul 28 19:31:46.961075 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:46.960930    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaa35347-b733-44aa-a2be-a3c7227a822b" containerName="payload-pre-processing"
Jul 28 19:31:46.961075 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:46.960937    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bd95977-49ef-4ea9-bd3a-1d47abfcf9d2" containerName="maas-api"
Jul 28 19:31:46.965526 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:46.965508    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"
Jul 28 19:31:46.967991 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:46.967957    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:31:46.967991 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:46.967992    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-55d61f68-serving-cert\""
Jul 28 19:31:46.968787 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:46.968770    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z259w\""
Jul 28 19:31:46.968884 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:46.968805    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 19:31:46.973949 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:46.973929    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"]
Jul 28 19:31:47.066550 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.066522    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/546a8631-f6e0-4fb4-86c7-88403c4269be-openshift-service-ca\") pod \"maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf\" (UID: \"546a8631-f6e0-4fb4-86c7-88403c4269be\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"
Jul 28 19:31:47.066688 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.066597    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkk5c\" (UniqueName: \"kubernetes.io/projected/546a8631-f6e0-4fb4-86c7-88403c4269be-kube-api-access-qkk5c\") pod \"maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf\" (UID: \"546a8631-f6e0-4fb4-86c7-88403c4269be\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"
Jul 28 19:31:47.066688 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.066639    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/546a8631-f6e0-4fb4-86c7-88403c4269be-maas-api-tls\") pod \"maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf\" (UID: \"546a8631-f6e0-4fb4-86c7-88403c4269be\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"
Jul 28 19:31:47.167695 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.167662    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkk5c\" (UniqueName: \"kubernetes.io/projected/546a8631-f6e0-4fb4-86c7-88403c4269be-kube-api-access-qkk5c\") pod \"maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf\" (UID: \"546a8631-f6e0-4fb4-86c7-88403c4269be\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"
Jul 28 19:31:47.167864 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.167714    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/546a8631-f6e0-4fb4-86c7-88403c4269be-maas-api-tls\") pod \"maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf\" (UID: \"546a8631-f6e0-4fb4-86c7-88403c4269be\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"
Jul 28 19:31:47.167864 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.167747    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/546a8631-f6e0-4fb4-86c7-88403c4269be-openshift-service-ca\") pod \"maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf\" (UID: \"546a8631-f6e0-4fb4-86c7-88403c4269be\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"
Jul 28 19:31:47.168000 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:31:47.167882    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-serving-cert: secret "maas-api-e2e-derive-55d61f68-serving-cert" not found
Jul 28 19:31:47.168000 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:31:47.167962    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/546a8631-f6e0-4fb4-86c7-88403c4269be-maas-api-tls podName:546a8631-f6e0-4fb4-86c7-88403c4269be nodeName:}" failed. No retries permitted until 2026-07-28 19:31:47.667945653 +0000 UTC m=+1755.614430511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/546a8631-f6e0-4fb4-86c7-88403c4269be-maas-api-tls") pod "maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf" (UID: "546a8631-f6e0-4fb4-86c7-88403c4269be") : secret "maas-api-e2e-derive-55d61f68-serving-cert" not found
Jul 28 19:31:47.168294 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.168276    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/546a8631-f6e0-4fb4-86c7-88403c4269be-openshift-service-ca\") pod \"maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf\" (UID: \"546a8631-f6e0-4fb4-86c7-88403c4269be\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"
Jul 28 19:31:47.176862 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.176834    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkk5c\" (UniqueName: \"kubernetes.io/projected/546a8631-f6e0-4fb4-86c7-88403c4269be-kube-api-access-qkk5c\") pod \"maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf\" (UID: \"546a8631-f6e0-4fb4-86c7-88403c4269be\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"
Jul 28 19:31:47.678851 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.678816    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/546a8631-f6e0-4fb4-86c7-88403c4269be-maas-api-tls\") pod \"maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf\" (UID: \"546a8631-f6e0-4fb4-86c7-88403c4269be\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"
Jul 28 19:31:47.681190 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.681169    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/546a8631-f6e0-4fb4-86c7-88403c4269be-maas-api-tls\") pod \"maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf\" (UID: \"546a8631-f6e0-4fb4-86c7-88403c4269be\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"
Jul 28 19:31:47.688101 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.688079    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"]
Jul 28 19:31:47.691517 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.691497    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"
Jul 28 19:31:47.694498 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.694461    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-55d61f68-dockercfg-fwx9c\""
Jul 28 19:31:47.694498 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.694470    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-55d61f68\""
Jul 28 19:31:47.704084 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.704062    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"]
Jul 28 19:31:47.759755 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.759731    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"]
Jul 28 19:31:47.762823 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.762807    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"
Jul 28 19:31:47.776928 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.776907    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"]
Jul 28 19:31:47.780232 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.780212    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhj2k\" (UniqueName: \"kubernetes.io/projected/e9c55671-073c-4ba4-ac94-e763cee27539-kube-api-access-dhj2k\") pod \"payload-processing-e2e-derive-55d61f68-646f7f698-dsz64\" (UID: \"e9c55671-073c-4ba4-ac94-e763cee27539\") " pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"
Jul 28 19:31:47.780324 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.780272    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9c55671-073c-4ba4-ac94-e763cee27539-plugins-config-volume\") pod \"payload-processing-e2e-derive-55d61f68-646f7f698-dsz64\" (UID: \"e9c55671-073c-4ba4-ac94-e763cee27539\") " pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"
Jul 28 19:31:47.877110 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.877091    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"
Jul 28 19:31:47.880961 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.880940    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swcqx\" (UniqueName: \"kubernetes.io/projected/d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2-kube-api-access-swcqx\") pod \"payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4\" (UID: \"d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"
Jul 28 19:31:47.881030 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.880986    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9c55671-073c-4ba4-ac94-e763cee27539-plugins-config-volume\") pod \"payload-processing-e2e-derive-55d61f68-646f7f698-dsz64\" (UID: \"e9c55671-073c-4ba4-ac94-e763cee27539\") " pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"
Jul 28 19:31:47.881081 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.881062    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4\" (UID: \"d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"
Jul 28 19:31:47.881158 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.881138    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhj2k\" (UniqueName: \"kubernetes.io/projected/e9c55671-073c-4ba4-ac94-e763cee27539-kube-api-access-dhj2k\") pod \"payload-processing-e2e-derive-55d61f68-646f7f698-dsz64\" (UID: \"e9c55671-073c-4ba4-ac94-e763cee27539\") " pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"
Jul 28 19:31:47.881634 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.881619    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9c55671-073c-4ba4-ac94-e763cee27539-plugins-config-volume\") pod \"payload-processing-e2e-derive-55d61f68-646f7f698-dsz64\" (UID: \"e9c55671-073c-4ba4-ac94-e763cee27539\") " pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"
Jul 28 19:31:47.889429 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.889402    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhj2k\" (UniqueName: \"kubernetes.io/projected/e9c55671-073c-4ba4-ac94-e763cee27539-kube-api-access-dhj2k\") pod \"payload-processing-e2e-derive-55d61f68-646f7f698-dsz64\" (UID: \"e9c55671-073c-4ba4-ac94-e763cee27539\") " pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"
Jul 28 19:31:47.981929 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.981885    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swcqx\" (UniqueName: \"kubernetes.io/projected/d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2-kube-api-access-swcqx\") pod \"payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4\" (UID: \"d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"
Jul 28 19:31:47.982243 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.982040    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4\" (UID: \"d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"
Jul 28 19:31:47.982969 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.982946    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4\" (UID: \"d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"
Jul 28 19:31:47.990833 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:47.990811    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swcqx\" (UniqueName: \"kubernetes.io/projected/d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2-kube-api-access-swcqx\") pod \"payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4\" (UID: \"d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"
Jul 28 19:31:48.002650 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.002626    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"
Jul 28 19:31:48.009122 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.009099    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"]
Jul 28 19:31:48.013135 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:31:48.013112    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod546a8631_f6e0_4fb4_86c7_88403c4269be.slice/crio-31d8593588a0dd7044ae3c4d7a1df888467ca2c9f367c973d77b570681fab076 WatchSource:0}: Error finding container 31d8593588a0dd7044ae3c4d7a1df888467ca2c9f367c973d77b570681fab076: Status 404 returned error can't find the container with id 31d8593588a0dd7044ae3c4d7a1df888467ca2c9f367c973d77b570681fab076
Jul 28 19:31:48.073296 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.073267    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"
Jul 28 19:31:48.140558 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.140451    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"]
Jul 28 19:31:48.145268 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:31:48.145240    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9c55671_073c_4ba4_ac94_e763cee27539.slice/crio-80e1e177fe653fa839afe92a09fd0ca7a94662f92cd80f92ee5822c8a38ab837 WatchSource:0}: Error finding container 80e1e177fe653fa839afe92a09fd0ca7a94662f92cd80f92ee5822c8a38ab837: Status 404 returned error can't find the container with id 80e1e177fe653fa839afe92a09fd0ca7a94662f92cd80f92ee5822c8a38ab837
Jul 28 19:31:48.220691 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.220664    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"]
Jul 28 19:31:48.223181 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:31:48.223157    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd38c14b6_f3c3_4bc8_9860_0e9e4d0b47f2.slice/crio-6d9ce8d5cc02c79b04ef4eb4c16b47818ba6286917d221b68b1c418145821e3d WatchSource:0}: Error finding container 6d9ce8d5cc02c79b04ef4eb4c16b47818ba6286917d221b68b1c418145821e3d: Status 404 returned error can't find the container with id 6d9ce8d5cc02c79b04ef4eb4c16b47818ba6286917d221b68b1c418145821e3d
Jul 28 19:31:48.771522 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.771493    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4" event={"ID":"d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2","Type":"ContainerStarted","Data":"b1c2c219fec192cb133a5db0391703c65b84ef9ae4f43e897426159da886088f"}
Jul 28 19:31:48.771647 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.771554    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4" event={"ID":"d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2","Type":"ContainerStarted","Data":"6d9ce8d5cc02c79b04ef4eb4c16b47818ba6286917d221b68b1c418145821e3d"}
Jul 28 19:31:48.771647 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.771584    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"
Jul 28 19:31:48.772994 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.772971    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64" event={"ID":"e9c55671-073c-4ba4-ac94-e763cee27539","Type":"ContainerStarted","Data":"e699e9a3dfb9999391bc9e23b3c1329052b19d0b51fdc23e11d065b18861072c"}
Jul 28 19:31:48.773097 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.773018    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64" event={"ID":"e9c55671-073c-4ba4-ac94-e763cee27539","Type":"ContainerStarted","Data":"80e1e177fe653fa839afe92a09fd0ca7a94662f92cd80f92ee5822c8a38ab837"}
Jul 28 19:31:48.773097 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.773039    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"
Jul 28 19:31:48.774405 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.774384    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf" event={"ID":"546a8631-f6e0-4fb4-86c7-88403c4269be","Type":"ContainerStarted","Data":"df050e061bdb3c4b009fefc3a94fa61de6a6cb4fd0569824a8aa3470eda5ae4c"}
Jul 28 19:31:48.774535 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.774425    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf" event={"ID":"546a8631-f6e0-4fb4-86c7-88403c4269be","Type":"ContainerStarted","Data":"31d8593588a0dd7044ae3c4d7a1df888467ca2c9f367c973d77b570681fab076"}
Jul 28 19:31:48.774535 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.774454    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"
Jul 28 19:31:48.804565 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.804527    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4" podStartSLOduration=1.804513933 podStartE2EDuration="1.804513933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:31:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:31:48.802978339 +0000 UTC m=+1756.749463231" watchObservedRunningTime="2026-07-28 19:31:48.804513933 +0000 UTC m=+1756.750998800"
Jul 28 19:31:48.839606 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.839566    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf" podStartSLOduration=2.198450941 podStartE2EDuration="2.839551828s" totalImagesPullingTime="641.100887ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:31:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:31:48.83876144 +0000 UTC m=+1756.785246331" watchObservedRunningTime="2026-07-28 19:31:48.839551828 +0000 UTC m=+1756.786036695"
Jul 28 19:31:48.879767 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:48.879723    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64" podStartSLOduration=1.879711596 podStartE2EDuration="1.879711596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:31:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:31:48.87890267 +0000 UTC m=+1756.825387558" watchObservedRunningTime="2026-07-28 19:31:48.879711596 +0000 UTC m=+1756.826196461"
Jul 28 19:31:54.784598 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:54.784572    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"
Jul 28 19:31:59.781454 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:59.781417    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"
Jul 28 19:31:59.782097 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:31:59.781763    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"
Jul 28 19:32:27.572396 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.572318    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"]
Jul 28 19:32:27.572798 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.572544    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf" podUID="546a8631-f6e0-4fb4-86c7-88403c4269be" containerName="maas-api" containerID="cri-o://df050e061bdb3c4b009fefc3a94fa61de6a6cb4fd0569824a8aa3470eda5ae4c" gracePeriod=30
Jul 28 19:32:27.626180 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.626153    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"]
Jul 28 19:32:27.626417 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.626393    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64" podUID="e9c55671-073c-4ba4-ac94-e763cee27539" containerName="payload-processing" containerID="cri-o://e699e9a3dfb9999391bc9e23b3c1329052b19d0b51fdc23e11d065b18861072c" gracePeriod=30
Jul 28 19:32:27.639690 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.639664    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"]
Jul 28 19:32:27.639895 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.639870    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4" podUID="d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2" containerName="payload-pre-processing" containerID="cri-o://b1c2c219fec192cb133a5db0391703c65b84ef9ae4f43e897426159da886088f" gracePeriod=30
Jul 28 19:32:27.886176 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.886151    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"
Jul 28 19:32:27.924737 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.924710    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"
Jul 28 19:32:27.939717 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.939692    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"
Jul 28 19:32:27.964063 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.963983    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4" event={"ID":"d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2","Type":"ContainerDied","Data":"b1c2c219fec192cb133a5db0391703c65b84ef9ae4f43e897426159da886088f"}
Jul 28 19:32:27.964063 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.963883    2561 generic.go:356] "Generic (PLEG): container finished" podID="d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2" containerID="b1c2c219fec192cb133a5db0391703c65b84ef9ae4f43e897426159da886088f" exitCode=0
Jul 28 19:32:27.964697 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.964066    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4" event={"ID":"d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2","Type":"ContainerDied","Data":"6d9ce8d5cc02c79b04ef4eb4c16b47818ba6286917d221b68b1c418145821e3d"}
Jul 28 19:32:27.964697 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.964099    2561 scope.go:117] "RemoveContainer" containerID="b1c2c219fec192cb133a5db0391703c65b84ef9ae4f43e897426159da886088f"
Jul 28 19:32:27.964697 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.964009    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"
Jul 28 19:32:27.967050 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.967021    2561 generic.go:356] "Generic (PLEG): container finished" podID="e9c55671-073c-4ba4-ac94-e763cee27539" containerID="e699e9a3dfb9999391bc9e23b3c1329052b19d0b51fdc23e11d065b18861072c" exitCode=0
Jul 28 19:32:27.967159 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.967138    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"
Jul 28 19:32:27.967159 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.967147    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64" event={"ID":"e9c55671-073c-4ba4-ac94-e763cee27539","Type":"ContainerDied","Data":"e699e9a3dfb9999391bc9e23b3c1329052b19d0b51fdc23e11d065b18861072c"}
Jul 28 19:32:27.967559 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.967193    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64" event={"ID":"e9c55671-073c-4ba4-ac94-e763cee27539","Type":"ContainerDied","Data":"80e1e177fe653fa839afe92a09fd0ca7a94662f92cd80f92ee5822c8a38ab837"}
Jul 28 19:32:27.968733 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.968709    2561 generic.go:356] "Generic (PLEG): container finished" podID="546a8631-f6e0-4fb4-86c7-88403c4269be" containerID="df050e061bdb3c4b009fefc3a94fa61de6a6cb4fd0569824a8aa3470eda5ae4c" exitCode=0
Jul 28 19:32:27.968855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.968750    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf" event={"ID":"546a8631-f6e0-4fb4-86c7-88403c4269be","Type":"ContainerDied","Data":"df050e061bdb3c4b009fefc3a94fa61de6a6cb4fd0569824a8aa3470eda5ae4c"}
Jul 28 19:32:27.968855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.968755    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"
Jul 28 19:32:27.968855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.968789    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf" event={"ID":"546a8631-f6e0-4fb4-86c7-88403c4269be","Type":"ContainerDied","Data":"31d8593588a0dd7044ae3c4d7a1df888467ca2c9f367c973d77b570681fab076"}
Jul 28 19:32:27.974247 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.974231    2561 scope.go:117] "RemoveContainer" containerID="b1c2c219fec192cb133a5db0391703c65b84ef9ae4f43e897426159da886088f"
Jul 28 19:32:27.974546 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:32:27.974522    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1c2c219fec192cb133a5db0391703c65b84ef9ae4f43e897426159da886088f\": container with ID starting with b1c2c219fec192cb133a5db0391703c65b84ef9ae4f43e897426159da886088f not found: ID does not exist" containerID="b1c2c219fec192cb133a5db0391703c65b84ef9ae4f43e897426159da886088f"
Jul 28 19:32:27.974602 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.974559    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1c2c219fec192cb133a5db0391703c65b84ef9ae4f43e897426159da886088f"} err="failed to get container status \"b1c2c219fec192cb133a5db0391703c65b84ef9ae4f43e897426159da886088f\": rpc error: code = NotFound desc = could not find container \"b1c2c219fec192cb133a5db0391703c65b84ef9ae4f43e897426159da886088f\": container with ID starting with b1c2c219fec192cb133a5db0391703c65b84ef9ae4f43e897426159da886088f not found: ID does not exist"
Jul 28 19:32:27.974602 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.974583    2561 scope.go:117] "RemoveContainer" containerID="e699e9a3dfb9999391bc9e23b3c1329052b19d0b51fdc23e11d065b18861072c"
Jul 28 19:32:27.982123 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.982107    2561 scope.go:117] "RemoveContainer" containerID="e699e9a3dfb9999391bc9e23b3c1329052b19d0b51fdc23e11d065b18861072c"
Jul 28 19:32:27.982420 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:32:27.982393    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e699e9a3dfb9999391bc9e23b3c1329052b19d0b51fdc23e11d065b18861072c\": container with ID starting with e699e9a3dfb9999391bc9e23b3c1329052b19d0b51fdc23e11d065b18861072c not found: ID does not exist" containerID="e699e9a3dfb9999391bc9e23b3c1329052b19d0b51fdc23e11d065b18861072c"
Jul 28 19:32:27.982465 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.982427    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e699e9a3dfb9999391bc9e23b3c1329052b19d0b51fdc23e11d065b18861072c"} err="failed to get container status \"e699e9a3dfb9999391bc9e23b3c1329052b19d0b51fdc23e11d065b18861072c\": rpc error: code = NotFound desc = could not find container \"e699e9a3dfb9999391bc9e23b3c1329052b19d0b51fdc23e11d065b18861072c\": container with ID starting with e699e9a3dfb9999391bc9e23b3c1329052b19d0b51fdc23e11d065b18861072c not found: ID does not exist"
Jul 28 19:32:27.982465 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.982442    2561 scope.go:117] "RemoveContainer" containerID="df050e061bdb3c4b009fefc3a94fa61de6a6cb4fd0569824a8aa3470eda5ae4c"
Jul 28 19:32:27.989703 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.989685    2561 scope.go:117] "RemoveContainer" containerID="df050e061bdb3c4b009fefc3a94fa61de6a6cb4fd0569824a8aa3470eda5ae4c"
Jul 28 19:32:27.989972 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:32:27.989955    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df050e061bdb3c4b009fefc3a94fa61de6a6cb4fd0569824a8aa3470eda5ae4c\": container with ID starting with df050e061bdb3c4b009fefc3a94fa61de6a6cb4fd0569824a8aa3470eda5ae4c not found: ID does not exist" containerID="df050e061bdb3c4b009fefc3a94fa61de6a6cb4fd0569824a8aa3470eda5ae4c"
Jul 28 19:32:27.990022 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:27.989977    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df050e061bdb3c4b009fefc3a94fa61de6a6cb4fd0569824a8aa3470eda5ae4c"} err="failed to get container status \"df050e061bdb3c4b009fefc3a94fa61de6a6cb4fd0569824a8aa3470eda5ae4c\": rpc error: code = NotFound desc = could not find container \"df050e061bdb3c4b009fefc3a94fa61de6a6cb4fd0569824a8aa3470eda5ae4c\": container with ID starting with df050e061bdb3c4b009fefc3a94fa61de6a6cb4fd0569824a8aa3470eda5ae4c not found: ID does not exist"
Jul 28 19:32:28.003914 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.003896    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhj2k\" (UniqueName: \"kubernetes.io/projected/e9c55671-073c-4ba4-ac94-e763cee27539-kube-api-access-dhj2k\") pod \"e9c55671-073c-4ba4-ac94-e763cee27539\" (UID: \"e9c55671-073c-4ba4-ac94-e763cee27539\") "
Jul 28 19:32:28.004008 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.003929    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9c55671-073c-4ba4-ac94-e763cee27539-plugins-config-volume\") pod \"e9c55671-073c-4ba4-ac94-e763cee27539\" (UID: \"e9c55671-073c-4ba4-ac94-e763cee27539\") "
Jul 28 19:32:28.004285 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.004263    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9c55671-073c-4ba4-ac94-e763cee27539-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e9c55671-073c-4ba4-ac94-e763cee27539" (UID: "e9c55671-073c-4ba4-ac94-e763cee27539"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:32:28.005832 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.005812    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9c55671-073c-4ba4-ac94-e763cee27539-kube-api-access-dhj2k" (OuterVolumeSpecName: "kube-api-access-dhj2k") pod "e9c55671-073c-4ba4-ac94-e763cee27539" (UID: "e9c55671-073c-4ba4-ac94-e763cee27539"). InnerVolumeSpecName "kube-api-access-dhj2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:32:28.104666 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.104617    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkk5c\" (UniqueName: \"kubernetes.io/projected/546a8631-f6e0-4fb4-86c7-88403c4269be-kube-api-access-qkk5c\") pod \"546a8631-f6e0-4fb4-86c7-88403c4269be\" (UID: \"546a8631-f6e0-4fb4-86c7-88403c4269be\") "
Jul 28 19:32:28.104666 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.104660    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/546a8631-f6e0-4fb4-86c7-88403c4269be-maas-api-tls\") pod \"546a8631-f6e0-4fb4-86c7-88403c4269be\" (UID: \"546a8631-f6e0-4fb4-86c7-88403c4269be\") "
Jul 28 19:32:28.104783 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.104745    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/546a8631-f6e0-4fb4-86c7-88403c4269be-openshift-service-ca\") pod \"546a8631-f6e0-4fb4-86c7-88403c4269be\" (UID: \"546a8631-f6e0-4fb4-86c7-88403c4269be\") "
Jul 28 19:32:28.104820 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.104798    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swcqx\" (UniqueName: \"kubernetes.io/projected/d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2-kube-api-access-swcqx\") pod \"d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2\" (UID: \"d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2\") "
Jul 28 19:32:28.104866 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.104830    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2-plugins-config-volume\") pod \"d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2\" (UID: \"d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2\") "
Jul 28 19:32:28.105165 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.105145    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhj2k\" (UniqueName: \"kubernetes.io/projected/e9c55671-073c-4ba4-ac94-e763cee27539-kube-api-access-dhj2k\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:32:28.105263 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.105170    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9c55671-073c-4ba4-ac94-e763cee27539-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:32:28.105263 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.105168    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/546a8631-f6e0-4fb4-86c7-88403c4269be-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "546a8631-f6e0-4fb4-86c7-88403c4269be" (UID: "546a8631-f6e0-4fb4-86c7-88403c4269be"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:32:28.105263 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.105247    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2" (UID: "d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:32:28.106694 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.106672    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/546a8631-f6e0-4fb4-86c7-88403c4269be-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "546a8631-f6e0-4fb4-86c7-88403c4269be" (UID: "546a8631-f6e0-4fb4-86c7-88403c4269be"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:32:28.106765 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.106700    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/546a8631-f6e0-4fb4-86c7-88403c4269be-kube-api-access-qkk5c" (OuterVolumeSpecName: "kube-api-access-qkk5c") pod "546a8631-f6e0-4fb4-86c7-88403c4269be" (UID: "546a8631-f6e0-4fb4-86c7-88403c4269be"). InnerVolumeSpecName "kube-api-access-qkk5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:32:28.106812 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.106767    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2-kube-api-access-swcqx" (OuterVolumeSpecName: "kube-api-access-swcqx") pod "d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2" (UID: "d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2"). InnerVolumeSpecName "kube-api-access-swcqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:32:28.205862 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.205841    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/546a8631-f6e0-4fb4-86c7-88403c4269be-openshift-service-ca\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:32:28.205959 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.205866    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swcqx\" (UniqueName: \"kubernetes.io/projected/d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2-kube-api-access-swcqx\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:32:28.205959 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.205877    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:32:28.205959 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.205886    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkk5c\" (UniqueName: \"kubernetes.io/projected/546a8631-f6e0-4fb4-86c7-88403c4269be-kube-api-access-qkk5c\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:32:28.205959 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.205895    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/546a8631-f6e0-4fb4-86c7-88403c4269be-maas-api-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:32:28.296865 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.296839    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"]
Jul 28 19:32:28.314368 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.314342    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-55d61f68-95f9c6c8d-bnhw4"]
Jul 28 19:32:28.343239 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.343216    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"]
Jul 28 19:32:28.352498 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.352466    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-55d61f68-646f7f698-dsz64"]
Jul 28 19:32:28.378251 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.378195    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"]
Jul 28 19:32:28.385581 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.385563    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-55d61f68-5c45df4b48-ngsnf"]
Jul 28 19:32:28.604399 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.604374    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="546a8631-f6e0-4fb4-86c7-88403c4269be" path="/var/lib/kubelet/pods/546a8631-f6e0-4fb4-86c7-88403c4269be/volumes"
Jul 28 19:32:28.604748 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.604732    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2" path="/var/lib/kubelet/pods/d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2/volumes"
Jul 28 19:32:28.605063 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:28.605049    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9c55671-073c-4ba4-ac94-e763cee27539" path="/var/lib/kubelet/pods/e9c55671-073c-4ba4-ac94-e763cee27539/volumes"
Jul 28 19:32:32.554920 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:32.554892    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:32:32.557995 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:32.557971    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:32:32.561642 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:32.561612    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:32:32.565064 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:32:32.565047    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:34:13.180961 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.180863    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"]
Jul 28 19:34:13.181609 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.181361    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2" containerName="payload-pre-processing"
Jul 28 19:34:13.181609 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.181377    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2" containerName="payload-pre-processing"
Jul 28 19:34:13.181609 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.181396    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="546a8631-f6e0-4fb4-86c7-88403c4269be" containerName="maas-api"
Jul 28 19:34:13.181609 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.181405    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="546a8631-f6e0-4fb4-86c7-88403c4269be" containerName="maas-api"
Jul 28 19:34:13.181609 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.181427    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c55671-073c-4ba4-ac94-e763cee27539" containerName="payload-processing"
Jul 28 19:34:13.181609 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.181436    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c55671-073c-4ba4-ac94-e763cee27539" containerName="payload-processing"
Jul 28 19:34:13.181609 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.181559    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="546a8631-f6e0-4fb4-86c7-88403c4269be" containerName="maas-api"
Jul 28 19:34:13.181609 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.181574    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d38c14b6-f3c3-4bc8-9860-0e9e4d0b47f2" containerName="payload-pre-processing"
Jul 28 19:34:13.181609 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.181585    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9c55671-073c-4ba4-ac94-e763cee27539" containerName="payload-processing"
Jul 28 19:34:13.184991 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.184969    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"
Jul 28 19:34:13.187322 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.187296    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e72a09a2-serving-cert\""
Jul 28 19:34:13.187463 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.187340    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z259w\""
Jul 28 19:34:13.187652 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.187634    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:34:13.188439 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.188231    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 19:34:13.191559 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.191537    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"]
Jul 28 19:34:13.229637 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.229611    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b10bc8ef-1108-4678-99bf-ace532a938a9-maas-api-tls\") pod \"maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5\" (UID: \"b10bc8ef-1108-4678-99bf-ace532a938a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"
Jul 28 19:34:13.229812 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.229660    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57wht\" (UniqueName: \"kubernetes.io/projected/b10bc8ef-1108-4678-99bf-ace532a938a9-kube-api-access-57wht\") pod \"maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5\" (UID: \"b10bc8ef-1108-4678-99bf-ace532a938a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"
Jul 28 19:34:13.229812 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.229701    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b10bc8ef-1108-4678-99bf-ace532a938a9-openshift-service-ca\") pod \"maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5\" (UID: \"b10bc8ef-1108-4678-99bf-ace532a938a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"
Jul 28 19:34:13.330913 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.330877    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b10bc8ef-1108-4678-99bf-ace532a938a9-openshift-service-ca\") pod \"maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5\" (UID: \"b10bc8ef-1108-4678-99bf-ace532a938a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"
Jul 28 19:34:13.331120 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.331042    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b10bc8ef-1108-4678-99bf-ace532a938a9-maas-api-tls\") pod \"maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5\" (UID: \"b10bc8ef-1108-4678-99bf-ace532a938a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"
Jul 28 19:34:13.331202 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.331182    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57wht\" (UniqueName: \"kubernetes.io/projected/b10bc8ef-1108-4678-99bf-ace532a938a9-kube-api-access-57wht\") pod \"maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5\" (UID: \"b10bc8ef-1108-4678-99bf-ace532a938a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"
Jul 28 19:34:13.331276 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:34:13.331211    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-serving-cert: secret "maas-api-e2e-mt-e72a09a2-serving-cert" not found
Jul 28 19:34:13.331323 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:34:13.331296    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10bc8ef-1108-4678-99bf-ace532a938a9-maas-api-tls podName:b10bc8ef-1108-4678-99bf-ace532a938a9 nodeName:}" failed. No retries permitted until 2026-07-28 19:34:13.831274841 +0000 UTC m=+1901.777759701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b10bc8ef-1108-4678-99bf-ace532a938a9-maas-api-tls") pod "maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5" (UID: "b10bc8ef-1108-4678-99bf-ace532a938a9") : secret "maas-api-e2e-mt-e72a09a2-serving-cert" not found
Jul 28 19:34:13.331716 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.331683    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b10bc8ef-1108-4678-99bf-ace532a938a9-openshift-service-ca\") pod \"maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5\" (UID: \"b10bc8ef-1108-4678-99bf-ace532a938a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"
Jul 28 19:34:13.341508 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.341467    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57wht\" (UniqueName: \"kubernetes.io/projected/b10bc8ef-1108-4678-99bf-ace532a938a9-kube-api-access-57wht\") pod \"maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5\" (UID: \"b10bc8ef-1108-4678-99bf-ace532a938a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"
Jul 28 19:34:13.836453 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.836422    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b10bc8ef-1108-4678-99bf-ace532a938a9-maas-api-tls\") pod \"maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5\" (UID: \"b10bc8ef-1108-4678-99bf-ace532a938a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"
Jul 28 19:34:13.838991 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.838957    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b10bc8ef-1108-4678-99bf-ace532a938a9-maas-api-tls\") pod \"maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5\" (UID: \"b10bc8ef-1108-4678-99bf-ace532a938a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"
Jul 28 19:34:13.893338 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.893299    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"]
Jul 28 19:34:13.896906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.896885    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"
Jul 28 19:34:13.899141 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.899119    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e72a09a2\""
Jul 28 19:34:13.899261 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.899239    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e72a09a2-dockercfg-cq257\""
Jul 28 19:34:13.907082 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.907055    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"]
Jul 28 19:34:13.916820 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.916796    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"]
Jul 28 19:34:13.920328 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.920307    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"
Jul 28 19:34:13.930147 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.930110    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"]
Jul 28 19:34:13.937589 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.937558    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b542563-ebe8-49d1-8ad0-2b20094d7905-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp\" (UID: \"1b542563-ebe8-49d1-8ad0-2b20094d7905\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"
Jul 28 19:34:13.937708 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.937685    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37c03318-d16c-452e-ac35-d81024978139-plugins-config-volume\") pod \"payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m\" (UID: \"37c03318-d16c-452e-ac35-d81024978139\") " pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"
Jul 28 19:34:13.937756 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.937722    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw9g9\" (UniqueName: \"kubernetes.io/projected/37c03318-d16c-452e-ac35-d81024978139-kube-api-access-rw9g9\") pod \"payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m\" (UID: \"37c03318-d16c-452e-ac35-d81024978139\") " pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"
Jul 28 19:34:13.937791 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:13.937762    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp8ls\" (UniqueName: \"kubernetes.io/projected/1b542563-ebe8-49d1-8ad0-2b20094d7905-kube-api-access-gp8ls\") pod \"payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp\" (UID: \"1b542563-ebe8-49d1-8ad0-2b20094d7905\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"
Jul 28 19:34:14.038552 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.038524    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp8ls\" (UniqueName: \"kubernetes.io/projected/1b542563-ebe8-49d1-8ad0-2b20094d7905-kube-api-access-gp8ls\") pod \"payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp\" (UID: \"1b542563-ebe8-49d1-8ad0-2b20094d7905\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"
Jul 28 19:34:14.038717 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.038561    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b542563-ebe8-49d1-8ad0-2b20094d7905-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp\" (UID: \"1b542563-ebe8-49d1-8ad0-2b20094d7905\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"
Jul 28 19:34:14.038717 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.038647    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37c03318-d16c-452e-ac35-d81024978139-plugins-config-volume\") pod \"payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m\" (UID: \"37c03318-d16c-452e-ac35-d81024978139\") " pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"
Jul 28 19:34:14.038717 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.038664    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw9g9\" (UniqueName: \"kubernetes.io/projected/37c03318-d16c-452e-ac35-d81024978139-kube-api-access-rw9g9\") pod \"payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m\" (UID: \"37c03318-d16c-452e-ac35-d81024978139\") " pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"
Jul 28 19:34:14.039404 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.039375    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37c03318-d16c-452e-ac35-d81024978139-plugins-config-volume\") pod \"payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m\" (UID: \"37c03318-d16c-452e-ac35-d81024978139\") " pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"
Jul 28 19:34:14.039443 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.039415    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b542563-ebe8-49d1-8ad0-2b20094d7905-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp\" (UID: \"1b542563-ebe8-49d1-8ad0-2b20094d7905\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"
Jul 28 19:34:14.046036 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.046013    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw9g9\" (UniqueName: \"kubernetes.io/projected/37c03318-d16c-452e-ac35-d81024978139-kube-api-access-rw9g9\") pod \"payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m\" (UID: \"37c03318-d16c-452e-ac35-d81024978139\") " pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"
Jul 28 19:34:14.046254 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.046233    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp8ls\" (UniqueName: \"kubernetes.io/projected/1b542563-ebe8-49d1-8ad0-2b20094d7905-kube-api-access-gp8ls\") pod \"payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp\" (UID: \"1b542563-ebe8-49d1-8ad0-2b20094d7905\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"
Jul 28 19:34:14.099695 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.099618    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"
Jul 28 19:34:14.209078 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.208823    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"
Jul 28 19:34:14.232082 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.232002    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"
Jul 28 19:34:14.245194 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.245168    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"]
Jul 28 19:34:14.248449 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:34:14.248415    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb10bc8ef_1108_4678_99bf_ace532a938a9.slice/crio-0f4f9a34b61251a9189530e91ccf3287d09c7cea35ce2b027ae169af5d30095a WatchSource:0}: Error finding container 0f4f9a34b61251a9189530e91ccf3287d09c7cea35ce2b027ae169af5d30095a: Status 404 returned error can't find the container with id 0f4f9a34b61251a9189530e91ccf3287d09c7cea35ce2b027ae169af5d30095a
Jul 28 19:34:14.371679 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.371655    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"]
Jul 28 19:34:14.374952 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:34:14.374906    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b542563_ebe8_49d1_8ad0_2b20094d7905.slice/crio-453ee280107abc0c9c5a18761ef808fb2b4ea88533ed14b581eb1d47a9bcff46 WatchSource:0}: Error finding container 453ee280107abc0c9c5a18761ef808fb2b4ea88533ed14b581eb1d47a9bcff46: Status 404 returned error can't find the container with id 453ee280107abc0c9c5a18761ef808fb2b4ea88533ed14b581eb1d47a9bcff46
Jul 28 19:34:14.397259 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.397238    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"]
Jul 28 19:34:14.400072 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:34:14.400042    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37c03318_d16c_452e_ac35_d81024978139.slice/crio-8268d8a519d43355f373d9cf68c9981886b0146e689d3a983975466b242a1902 WatchSource:0}: Error finding container 8268d8a519d43355f373d9cf68c9981886b0146e689d3a983975466b242a1902: Status 404 returned error can't find the container with id 8268d8a519d43355f373d9cf68c9981886b0146e689d3a983975466b242a1902
Jul 28 19:34:14.462871 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.462834    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp" event={"ID":"1b542563-ebe8-49d1-8ad0-2b20094d7905","Type":"ContainerStarted","Data":"453ee280107abc0c9c5a18761ef808fb2b4ea88533ed14b581eb1d47a9bcff46"}
Jul 28 19:34:14.464213 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.464175    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5" event={"ID":"b10bc8ef-1108-4678-99bf-ace532a938a9","Type":"ContainerStarted","Data":"0f4f9a34b61251a9189530e91ccf3287d09c7cea35ce2b027ae169af5d30095a"}
Jul 28 19:34:14.465630 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:14.465597    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m" event={"ID":"37c03318-d16c-452e-ac35-d81024978139","Type":"ContainerStarted","Data":"8268d8a519d43355f373d9cf68c9981886b0146e689d3a983975466b242a1902"}
Jul 28 19:34:15.472499 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:15.472449    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp" event={"ID":"1b542563-ebe8-49d1-8ad0-2b20094d7905","Type":"ContainerStarted","Data":"3f3b0891339e13ab4b9b3f64e7f9a00331c9f3be3849ba1a79a66cb958a24912"}
Jul 28 19:34:15.472951 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:15.472579    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"
Jul 28 19:34:15.474308 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:15.474280    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5" event={"ID":"b10bc8ef-1108-4678-99bf-ace532a938a9","Type":"ContainerStarted","Data":"cf595b086517710d030b7ec75780aacd58e884bf62522b22ed791f56b930ec83"}
Jul 28 19:34:15.474449 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:15.474332    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"
Jul 28 19:34:15.475959 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:15.475933    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m" event={"ID":"37c03318-d16c-452e-ac35-d81024978139","Type":"ContainerStarted","Data":"3dfa1f46724ac2684f55a058a1fa8a4056eed828f94e55d7f00b58a2e9fcc0f0"}
Jul 28 19:34:15.476079 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:15.475989    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"
Jul 28 19:34:15.487904 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:15.487863    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp" podStartSLOduration=2.487848706 podStartE2EDuration="2.487848706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:34:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:15.486635908 +0000 UTC m=+1903.433120800" watchObservedRunningTime="2026-07-28 19:34:15.487848706 +0000 UTC m=+1903.434333575"
Jul 28 19:34:15.502175 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:15.502132    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m" podStartSLOduration=2.502118121 podStartE2EDuration="2.502118121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:34:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:15.500477584 +0000 UTC m=+1903.446962472" watchObservedRunningTime="2026-07-28 19:34:15.502118121 +0000 UTC m=+1903.448602988"
Jul 28 19:34:21.485266 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:21.485239    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"
Jul 28 19:34:21.501294 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:21.501251    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5" podStartSLOduration=8.076286186 podStartE2EDuration="8.501236357s" totalImagesPullingTime="424.950171ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:34:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:34:15.519496047 +0000 UTC m=+1903.465980909" watchObservedRunningTime="2026-07-28 19:34:21.501236357 +0000 UTC m=+1909.447721265"
Jul 28 19:34:26.482337 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:26.482299    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"
Jul 28 19:34:26.482758 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:26.482374    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"
Jul 28 19:34:55.435393 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.435361    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"]
Jul 28 19:34:55.435942 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.435623    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5" podUID="b10bc8ef-1108-4678-99bf-ace532a938a9" containerName="maas-api" containerID="cri-o://cf595b086517710d030b7ec75780aacd58e884bf62522b22ed791f56b930ec83" gracePeriod=30
Jul 28 19:34:55.480323 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.480295    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"]
Jul 28 19:34:55.480573 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.480539    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m" podUID="37c03318-d16c-452e-ac35-d81024978139" containerName="payload-processing" containerID="cri-o://3dfa1f46724ac2684f55a058a1fa8a4056eed828f94e55d7f00b58a2e9fcc0f0" gracePeriod=30
Jul 28 19:34:55.489182 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.489156    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"]
Jul 28 19:34:55.489554 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.489526    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp" podUID="1b542563-ebe8-49d1-8ad0-2b20094d7905" containerName="payload-pre-processing" containerID="cri-o://3f3b0891339e13ab4b9b3f64e7f9a00331c9f3be3849ba1a79a66cb958a24912" gracePeriod=30
Jul 28 19:34:55.665178 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.665148    2561 generic.go:356] "Generic (PLEG): container finished" podID="1b542563-ebe8-49d1-8ad0-2b20094d7905" containerID="3f3b0891339e13ab4b9b3f64e7f9a00331c9f3be3849ba1a79a66cb958a24912" exitCode=0
Jul 28 19:34:55.665345 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.665219    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp" event={"ID":"1b542563-ebe8-49d1-8ad0-2b20094d7905","Type":"ContainerDied","Data":"3f3b0891339e13ab4b9b3f64e7f9a00331c9f3be3849ba1a79a66cb958a24912"}
Jul 28 19:34:55.666822 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.666797    2561 generic.go:356] "Generic (PLEG): container finished" podID="b10bc8ef-1108-4678-99bf-ace532a938a9" containerID="cf595b086517710d030b7ec75780aacd58e884bf62522b22ed791f56b930ec83" exitCode=0
Jul 28 19:34:55.666950 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.666887    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5" event={"ID":"b10bc8ef-1108-4678-99bf-ace532a938a9","Type":"ContainerDied","Data":"cf595b086517710d030b7ec75780aacd58e884bf62522b22ed791f56b930ec83"}
Jul 28 19:34:55.668330 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.668309    2561 generic.go:356] "Generic (PLEG): container finished" podID="37c03318-d16c-452e-ac35-d81024978139" containerID="3dfa1f46724ac2684f55a058a1fa8a4056eed828f94e55d7f00b58a2e9fcc0f0" exitCode=0
Jul 28 19:34:55.668438 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.668396    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m" event={"ID":"37c03318-d16c-452e-ac35-d81024978139","Type":"ContainerDied","Data":"3dfa1f46724ac2684f55a058a1fa8a4056eed828f94e55d7f00b58a2e9fcc0f0"}
Jul 28 19:34:55.758249 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.758226    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"
Jul 28 19:34:55.774219 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.774195    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"
Jul 28 19:34:55.779204 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.779186    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"
Jul 28 19:34:55.883170 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.883141    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rw9g9\" (UniqueName: \"kubernetes.io/projected/37c03318-d16c-452e-ac35-d81024978139-kube-api-access-rw9g9\") pod \"37c03318-d16c-452e-ac35-d81024978139\" (UID: \"37c03318-d16c-452e-ac35-d81024978139\") "
Jul 28 19:34:55.883309 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.883212    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37c03318-d16c-452e-ac35-d81024978139-plugins-config-volume\") pod \"37c03318-d16c-452e-ac35-d81024978139\" (UID: \"37c03318-d16c-452e-ac35-d81024978139\") "
Jul 28 19:34:55.883309 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.883278    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b10bc8ef-1108-4678-99bf-ace532a938a9-openshift-service-ca\") pod \"b10bc8ef-1108-4678-99bf-ace532a938a9\" (UID: \"b10bc8ef-1108-4678-99bf-ace532a938a9\") "
Jul 28 19:34:55.883434 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.883377    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b10bc8ef-1108-4678-99bf-ace532a938a9-maas-api-tls\") pod \"b10bc8ef-1108-4678-99bf-ace532a938a9\" (UID: \"b10bc8ef-1108-4678-99bf-ace532a938a9\") "
Jul 28 19:34:55.883526 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.883436    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b542563-ebe8-49d1-8ad0-2b20094d7905-plugins-config-volume\") pod \"1b542563-ebe8-49d1-8ad0-2b20094d7905\" (UID: \"1b542563-ebe8-49d1-8ad0-2b20094d7905\") "
Jul 28 19:34:55.883526 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.883457    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57wht\" (UniqueName: \"kubernetes.io/projected/b10bc8ef-1108-4678-99bf-ace532a938a9-kube-api-access-57wht\") pod \"b10bc8ef-1108-4678-99bf-ace532a938a9\" (UID: \"b10bc8ef-1108-4678-99bf-ace532a938a9\") "
Jul 28 19:34:55.883649 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.883524    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp8ls\" (UniqueName: \"kubernetes.io/projected/1b542563-ebe8-49d1-8ad0-2b20094d7905-kube-api-access-gp8ls\") pod \"1b542563-ebe8-49d1-8ad0-2b20094d7905\" (UID: \"1b542563-ebe8-49d1-8ad0-2b20094d7905\") "
Jul 28 19:34:55.883649 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.883630    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37c03318-d16c-452e-ac35-d81024978139-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "37c03318-d16c-452e-ac35-d81024978139" (UID: "37c03318-d16c-452e-ac35-d81024978139"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:34:55.883750 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.883645    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b10bc8ef-1108-4678-99bf-ace532a938a9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b10bc8ef-1108-4678-99bf-ace532a938a9" (UID: "b10bc8ef-1108-4678-99bf-ace532a938a9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:34:55.883871 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.883851    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b10bc8ef-1108-4678-99bf-ace532a938a9-openshift-service-ca\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:34:55.883933 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.883882    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37c03318-d16c-452e-ac35-d81024978139-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:34:55.883933 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.883854    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b542563-ebe8-49d1-8ad0-2b20094d7905-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1b542563-ebe8-49d1-8ad0-2b20094d7905" (UID: "1b542563-ebe8-49d1-8ad0-2b20094d7905"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:34:55.885313 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.885283    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37c03318-d16c-452e-ac35-d81024978139-kube-api-access-rw9g9" (OuterVolumeSpecName: "kube-api-access-rw9g9") pod "37c03318-d16c-452e-ac35-d81024978139" (UID: "37c03318-d16c-452e-ac35-d81024978139"). InnerVolumeSpecName "kube-api-access-rw9g9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:34:55.885529 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.885507    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b10bc8ef-1108-4678-99bf-ace532a938a9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b10bc8ef-1108-4678-99bf-ace532a938a9" (UID: "b10bc8ef-1108-4678-99bf-ace532a938a9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:34:55.885529 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.885514    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b10bc8ef-1108-4678-99bf-ace532a938a9-kube-api-access-57wht" (OuterVolumeSpecName: "kube-api-access-57wht") pod "b10bc8ef-1108-4678-99bf-ace532a938a9" (UID: "b10bc8ef-1108-4678-99bf-ace532a938a9"). InnerVolumeSpecName "kube-api-access-57wht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:34:55.885637 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.885616    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b542563-ebe8-49d1-8ad0-2b20094d7905-kube-api-access-gp8ls" (OuterVolumeSpecName: "kube-api-access-gp8ls") pod "1b542563-ebe8-49d1-8ad0-2b20094d7905" (UID: "1b542563-ebe8-49d1-8ad0-2b20094d7905"). InnerVolumeSpecName "kube-api-access-gp8ls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:34:55.985041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.985014    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57wht\" (UniqueName: \"kubernetes.io/projected/b10bc8ef-1108-4678-99bf-ace532a938a9-kube-api-access-57wht\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:34:55.985041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.985038    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp8ls\" (UniqueName: \"kubernetes.io/projected/1b542563-ebe8-49d1-8ad0-2b20094d7905-kube-api-access-gp8ls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:34:55.985199 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.985049    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rw9g9\" (UniqueName: \"kubernetes.io/projected/37c03318-d16c-452e-ac35-d81024978139-kube-api-access-rw9g9\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:34:55.985199 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.985059    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b10bc8ef-1108-4678-99bf-ace532a938a9-maas-api-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:34:55.985199 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:55.985068    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b542563-ebe8-49d1-8ad0-2b20094d7905-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:34:56.675908 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:56.675888    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"
Jul 28 19:34:56.676284 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:56.675887    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m" event={"ID":"37c03318-d16c-452e-ac35-d81024978139","Type":"ContainerDied","Data":"8268d8a519d43355f373d9cf68c9981886b0146e689d3a983975466b242a1902"}
Jul 28 19:34:56.676284 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:56.676007    2561 scope.go:117] "RemoveContainer" containerID="3dfa1f46724ac2684f55a058a1fa8a4056eed828f94e55d7f00b58a2e9fcc0f0"
Jul 28 19:34:56.677293 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:56.677276    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"
Jul 28 19:34:56.677380 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:56.677274    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp" event={"ID":"1b542563-ebe8-49d1-8ad0-2b20094d7905","Type":"ContainerDied","Data":"453ee280107abc0c9c5a18761ef808fb2b4ea88533ed14b581eb1d47a9bcff46"}
Jul 28 19:34:56.678879 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:56.678855    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5" event={"ID":"b10bc8ef-1108-4678-99bf-ace532a938a9","Type":"ContainerDied","Data":"0f4f9a34b61251a9189530e91ccf3287d09c7cea35ce2b027ae169af5d30095a"}
Jul 28 19:34:56.678978 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:56.678868    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"
Jul 28 19:34:56.685420 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:56.685399    2561 scope.go:117] "RemoveContainer" containerID="3f3b0891339e13ab4b9b3f64e7f9a00331c9f3be3849ba1a79a66cb958a24912"
Jul 28 19:34:56.692693 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:56.692673    2561 scope.go:117] "RemoveContainer" containerID="cf595b086517710d030b7ec75780aacd58e884bf62522b22ed791f56b930ec83"
Jul 28 19:34:56.700348 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:56.700329    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"]
Jul 28 19:34:56.703999 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:56.703979    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e72a09a2-dd9fc4d74-vmq8m"]
Jul 28 19:34:56.713647 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:56.713626    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"]
Jul 28 19:34:56.715725 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:56.715704    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e72a09a2-865bb989c4-hrkl5"]
Jul 28 19:34:56.725273 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:56.725251    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"]
Jul 28 19:34:56.728966 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:56.728947    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e72a09a2-8497d4d756-dwhjp"]
Jul 28 19:34:58.604144 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:58.604108    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b542563-ebe8-49d1-8ad0-2b20094d7905" path="/var/lib/kubelet/pods/1b542563-ebe8-49d1-8ad0-2b20094d7905/volumes"
Jul 28 19:34:58.604577 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:58.604469    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37c03318-d16c-452e-ac35-d81024978139" path="/var/lib/kubelet/pods/37c03318-d16c-452e-ac35-d81024978139/volumes"
Jul 28 19:34:58.604868 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:34:58.604851    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b10bc8ef-1108-4678-99bf-ace532a938a9" path="/var/lib/kubelet/pods/b10bc8ef-1108-4678-99bf-ace532a938a9/volumes"
Jul 28 19:35:24.637299 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.637223    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"]
Jul 28 19:35:24.637765 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.637742    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37c03318-d16c-452e-ac35-d81024978139" containerName="payload-processing"
Jul 28 19:35:24.637851 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.637768    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="37c03318-d16c-452e-ac35-d81024978139" containerName="payload-processing"
Jul 28 19:35:24.637851 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.637791    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b10bc8ef-1108-4678-99bf-ace532a938a9" containerName="maas-api"
Jul 28 19:35:24.637851 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.637801    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b10bc8ef-1108-4678-99bf-ace532a938a9" containerName="maas-api"
Jul 28 19:35:24.637851 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.637821    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b542563-ebe8-49d1-8ad0-2b20094d7905" containerName="payload-pre-processing"
Jul 28 19:35:24.637851 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.637830    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b542563-ebe8-49d1-8ad0-2b20094d7905" containerName="payload-pre-processing"
Jul 28 19:35:24.638002 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.637918    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b10bc8ef-1108-4678-99bf-ace532a938a9" containerName="maas-api"
Jul 28 19:35:24.638002 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.637930    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="37c03318-d16c-452e-ac35-d81024978139" containerName="payload-processing"
Jul 28 19:35:24.638002 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.637939    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b542563-ebe8-49d1-8ad0-2b20094d7905" containerName="payload-pre-processing"
Jul 28 19:35:24.641412 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.641390    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"
Jul 28 19:35:24.643978 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.643956    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-b965716e-serving-cert\""
Jul 28 19:35:24.644082 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.643964    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z259w\""
Jul 28 19:35:24.644082 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.643996    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:35:24.644878 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.644863    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 19:35:24.648892 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.648867    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"]
Jul 28 19:35:24.703995 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.703970    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e1a564-328e-4da3-8c52-0f675b68c8f4-maas-api-tls\") pod \"maas-api-e2e-mt-b965716e-664fd589cd-f7bx8\" (UID: \"60e1a564-328e-4da3-8c52-0f675b68c8f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"
Jul 28 19:35:24.704103 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.704017    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60e1a564-328e-4da3-8c52-0f675b68c8f4-openshift-service-ca\") pod \"maas-api-e2e-mt-b965716e-664fd589cd-f7bx8\" (UID: \"60e1a564-328e-4da3-8c52-0f675b68c8f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"
Jul 28 19:35:24.704103 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.704038    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4hwq\" (UniqueName: \"kubernetes.io/projected/60e1a564-328e-4da3-8c52-0f675b68c8f4-kube-api-access-m4hwq\") pod \"maas-api-e2e-mt-b965716e-664fd589cd-f7bx8\" (UID: \"60e1a564-328e-4da3-8c52-0f675b68c8f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"
Jul 28 19:35:24.804640 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.804619    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e1a564-328e-4da3-8c52-0f675b68c8f4-maas-api-tls\") pod \"maas-api-e2e-mt-b965716e-664fd589cd-f7bx8\" (UID: \"60e1a564-328e-4da3-8c52-0f675b68c8f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"
Jul 28 19:35:24.804743 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.804651    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60e1a564-328e-4da3-8c52-0f675b68c8f4-openshift-service-ca\") pod \"maas-api-e2e-mt-b965716e-664fd589cd-f7bx8\" (UID: \"60e1a564-328e-4da3-8c52-0f675b68c8f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"
Jul 28 19:35:24.804743 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.804669    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4hwq\" (UniqueName: \"kubernetes.io/projected/60e1a564-328e-4da3-8c52-0f675b68c8f4-kube-api-access-m4hwq\") pod \"maas-api-e2e-mt-b965716e-664fd589cd-f7bx8\" (UID: \"60e1a564-328e-4da3-8c52-0f675b68c8f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"
Jul 28 19:35:24.804844 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:35:24.804757    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-serving-cert: secret "maas-api-e2e-mt-b965716e-serving-cert" not found
Jul 28 19:35:24.804844 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:35:24.804832    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60e1a564-328e-4da3-8c52-0f675b68c8f4-maas-api-tls podName:60e1a564-328e-4da3-8c52-0f675b68c8f4 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:25.304814451 +0000 UTC m=+1973.251299313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/60e1a564-328e-4da3-8c52-0f675b68c8f4-maas-api-tls") pod "maas-api-e2e-mt-b965716e-664fd589cd-f7bx8" (UID: "60e1a564-328e-4da3-8c52-0f675b68c8f4") : secret "maas-api-e2e-mt-b965716e-serving-cert" not found
Jul 28 19:35:24.805288 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.805272    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60e1a564-328e-4da3-8c52-0f675b68c8f4-openshift-service-ca\") pod \"maas-api-e2e-mt-b965716e-664fd589cd-f7bx8\" (UID: \"60e1a564-328e-4da3-8c52-0f675b68c8f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"
Jul 28 19:35:24.815400 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:24.815377    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4hwq\" (UniqueName: \"kubernetes.io/projected/60e1a564-328e-4da3-8c52-0f675b68c8f4-kube-api-access-m4hwq\") pod \"maas-api-e2e-mt-b965716e-664fd589cd-f7bx8\" (UID: \"60e1a564-328e-4da3-8c52-0f675b68c8f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"
Jul 28 19:35:25.307355 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.307320    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e1a564-328e-4da3-8c52-0f675b68c8f4-maas-api-tls\") pod \"maas-api-e2e-mt-b965716e-664fd589cd-f7bx8\" (UID: \"60e1a564-328e-4da3-8c52-0f675b68c8f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"
Jul 28 19:35:25.309656 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.309633    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e1a564-328e-4da3-8c52-0f675b68c8f4-maas-api-tls\") pod \"maas-api-e2e-mt-b965716e-664fd589cd-f7bx8\" (UID: \"60e1a564-328e-4da3-8c52-0f675b68c8f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"
Jul 28 19:35:25.339805 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.339781    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"]
Jul 28 19:35:25.343250 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.343230    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"
Jul 28 19:35:25.345778 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.345759    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-b965716e\""
Jul 28 19:35:25.345875 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.345856    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-b965716e-dockercfg-gg25l\""
Jul 28 19:35:25.354011 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.353988    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"]
Jul 28 19:35:25.381179 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.381151    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"]
Jul 28 19:35:25.384394 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.384375    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"
Jul 28 19:35:25.393876 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.393855    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"]
Jul 28 19:35:25.408079 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.408055    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c68e901c-da46-493e-bbc3-1f68d6571616-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm\" (UID: \"c68e901c-da46-493e-bbc3-1f68d6571616\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"
Jul 28 19:35:25.408175 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.408098    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vp5p\" (UniqueName: \"kubernetes.io/projected/c68e901c-da46-493e-bbc3-1f68d6571616-kube-api-access-7vp5p\") pod \"payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm\" (UID: \"c68e901c-da46-493e-bbc3-1f68d6571616\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"
Jul 28 19:35:25.408175 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.408163    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2c4baf3-35e2-4f2d-b844-d33e870f1df3-plugins-config-volume\") pod \"payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm\" (UID: \"f2c4baf3-35e2-4f2d-b844-d33e870f1df3\") " pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"
Jul 28 19:35:25.408265 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.408181    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjsq9\" (UniqueName: \"kubernetes.io/projected/f2c4baf3-35e2-4f2d-b844-d33e870f1df3-kube-api-access-mjsq9\") pod \"payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm\" (UID: \"f2c4baf3-35e2-4f2d-b844-d33e870f1df3\") " pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"
Jul 28 19:35:25.509268 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.509233    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c68e901c-da46-493e-bbc3-1f68d6571616-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm\" (UID: \"c68e901c-da46-493e-bbc3-1f68d6571616\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"
Jul 28 19:35:25.509414 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.509285    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vp5p\" (UniqueName: \"kubernetes.io/projected/c68e901c-da46-493e-bbc3-1f68d6571616-kube-api-access-7vp5p\") pod \"payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm\" (UID: \"c68e901c-da46-493e-bbc3-1f68d6571616\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"
Jul 28 19:35:25.509414 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.509330    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2c4baf3-35e2-4f2d-b844-d33e870f1df3-plugins-config-volume\") pod \"payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm\" (UID: \"f2c4baf3-35e2-4f2d-b844-d33e870f1df3\") " pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"
Jul 28 19:35:25.509414 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.509351    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjsq9\" (UniqueName: \"kubernetes.io/projected/f2c4baf3-35e2-4f2d-b844-d33e870f1df3-kube-api-access-mjsq9\") pod \"payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm\" (UID: \"f2c4baf3-35e2-4f2d-b844-d33e870f1df3\") " pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"
Jul 28 19:35:25.510009 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.509989    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c68e901c-da46-493e-bbc3-1f68d6571616-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm\" (UID: \"c68e901c-da46-493e-bbc3-1f68d6571616\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"
Jul 28 19:35:25.510009 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.510005    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2c4baf3-35e2-4f2d-b844-d33e870f1df3-plugins-config-volume\") pod \"payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm\" (UID: \"f2c4baf3-35e2-4f2d-b844-d33e870f1df3\") " pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"
Jul 28 19:35:25.517838 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.517814    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjsq9\" (UniqueName: \"kubernetes.io/projected/f2c4baf3-35e2-4f2d-b844-d33e870f1df3-kube-api-access-mjsq9\") pod \"payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm\" (UID: \"f2c4baf3-35e2-4f2d-b844-d33e870f1df3\") " pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"
Jul 28 19:35:25.517941 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.517920    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vp5p\" (UniqueName: \"kubernetes.io/projected/c68e901c-da46-493e-bbc3-1f68d6571616-kube-api-access-7vp5p\") pod \"payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm\" (UID: \"c68e901c-da46-493e-bbc3-1f68d6571616\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"
Jul 28 19:35:25.553637 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.553614    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"
Jul 28 19:35:25.654846 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.654820    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"
Jul 28 19:35:25.679982 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.679952    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"]
Jul 28 19:35:25.680390 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:35:25.680361    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60e1a564_328e_4da3_8c52_0f675b68c8f4.slice/crio-315576e35eb0edf166b0086ac8fc747be320b1863103e5b9bcf2102f09bf23f3 WatchSource:0}: Error finding container 315576e35eb0edf166b0086ac8fc747be320b1863103e5b9bcf2102f09bf23f3: Status 404 returned error can't find the container with id 315576e35eb0edf166b0086ac8fc747be320b1863103e5b9bcf2102f09bf23f3
Jul 28 19:35:25.682348 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.682322    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:35:25.694701 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.694668    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"
Jul 28 19:35:25.795433 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.795400    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"]
Jul 28 19:35:25.800227 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:35:25.800201    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc68e901c_da46_493e_bbc3_1f68d6571616.slice/crio-0790cad8437e0ca3495eed531f6f8c96dcbb5cf259041e963f51c243f9d1d729 WatchSource:0}: Error finding container 0790cad8437e0ca3495eed531f6f8c96dcbb5cf259041e963f51c243f9d1d729: Status 404 returned error can't find the container with id 0790cad8437e0ca3495eed531f6f8c96dcbb5cf259041e963f51c243f9d1d729
Jul 28 19:35:25.820014 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.819983    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8" event={"ID":"60e1a564-328e-4da3-8c52-0f675b68c8f4","Type":"ContainerStarted","Data":"315576e35eb0edf166b0086ac8fc747be320b1863103e5b9bcf2102f09bf23f3"}
Jul 28 19:35:25.821851 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.821823    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm" event={"ID":"c68e901c-da46-493e-bbc3-1f68d6571616","Type":"ContainerStarted","Data":"0790cad8437e0ca3495eed531f6f8c96dcbb5cf259041e963f51c243f9d1d729"}
Jul 28 19:35:25.836978 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:25.836948    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"]
Jul 28 19:35:25.851675 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:35:25.851652    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2c4baf3_35e2_4f2d_b844_d33e870f1df3.slice/crio-8c6b79c748b8d91c625b72340540a6a1ce1f93cafc4bf5925c3d0bd2de944c6a WatchSource:0}: Error finding container 8c6b79c748b8d91c625b72340540a6a1ce1f93cafc4bf5925c3d0bd2de944c6a: Status 404 returned error can't find the container with id 8c6b79c748b8d91c625b72340540a6a1ce1f93cafc4bf5925c3d0bd2de944c6a
Jul 28 19:35:26.828266 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:26.828235    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8" event={"ID":"60e1a564-328e-4da3-8c52-0f675b68c8f4","Type":"ContainerStarted","Data":"6a15683e660b27cd710db43704010edbfb790a22ba8f5367dcce1f9ac7c05f4a"}
Jul 28 19:35:26.828786 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:26.828324    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"
Jul 28 19:35:26.829803 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:26.829778    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm" event={"ID":"f2c4baf3-35e2-4f2d-b844-d33e870f1df3","Type":"ContainerStarted","Data":"38f50a613785a9904a76cfc9ce9e425954624a86ff9e37d4e4067dab4f7f4fda"}
Jul 28 19:35:26.829918 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:26.829821    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm" event={"ID":"f2c4baf3-35e2-4f2d-b844-d33e870f1df3","Type":"ContainerStarted","Data":"8c6b79c748b8d91c625b72340540a6a1ce1f93cafc4bf5925c3d0bd2de944c6a"}
Jul 28 19:35:26.829918 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:26.829849    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"
Jul 28 19:35:26.831181 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:26.831163    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm" event={"ID":"c68e901c-da46-493e-bbc3-1f68d6571616","Type":"ContainerStarted","Data":"fc7a553bac2431215fbe0911d7f75f1a4c494b32bd83e9fe504daa029804c5a0"}
Jul 28 19:35:26.831289 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:26.831277    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"
Jul 28 19:35:26.844996 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:26.844955    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8" podStartSLOduration=2.471810773 podStartE2EDuration="2.844942453s" totalImagesPullingTime="373.13168ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:26.843950256 +0000 UTC m=+1974.790435152" watchObservedRunningTime="2026-07-28 19:35:26.844942453 +0000 UTC m=+1974.791427321"
Jul 28 19:35:26.859537 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:26.859496    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm" podStartSLOduration=1.859472421 podStartE2EDuration="1.859472421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:26.858104502 +0000 UTC m=+1974.804589396" watchObservedRunningTime="2026-07-28 19:35:26.859472421 +0000 UTC m=+1974.805957289"
Jul 28 19:35:26.871892 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:26.871849    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm" podStartSLOduration=1.871833232 podStartE2EDuration="1.871833232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:26.871751954 +0000 UTC m=+1974.818236825" watchObservedRunningTime="2026-07-28 19:35:26.871833232 +0000 UTC m=+1974.818318103"
Jul 28 19:35:32.841929 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:32.841907    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"
Jul 28 19:35:37.839367 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:37.839266    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"
Jul 28 19:35:37.841987 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:35:37.839430    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"
Jul 28 19:36:49.276857 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.276815    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"]
Jul 28 19:36:49.277791 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.277731    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8" podUID="60e1a564-328e-4da3-8c52-0f675b68c8f4" containerName="maas-api" containerID="cri-o://6a15683e660b27cd710db43704010edbfb790a22ba8f5367dcce1f9ac7c05f4a" gracePeriod=30
Jul 28 19:36:49.334897 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.334867    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"]
Jul 28 19:36:49.335324 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.335144    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm" podUID="f2c4baf3-35e2-4f2d-b844-d33e870f1df3" containerName="payload-processing" containerID="cri-o://38f50a613785a9904a76cfc9ce9e425954624a86ff9e37d4e4067dab4f7f4fda" gracePeriod=30
Jul 28 19:36:49.341745 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.341718    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"]
Jul 28 19:36:49.342071 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.341962    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm" podUID="c68e901c-da46-493e-bbc3-1f68d6571616" containerName="payload-pre-processing" containerID="cri-o://fc7a553bac2431215fbe0911d7f75f1a4c494b32bd83e9fe504daa029804c5a0" gracePeriod=30
Jul 28 19:36:49.593843 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.593820    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"
Jul 28 19:36:49.622759 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.622739    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"
Jul 28 19:36:49.629418 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.629390    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"
Jul 28 19:36:49.723093 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.723062    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e1a564-328e-4da3-8c52-0f675b68c8f4-maas-api-tls\") pod \"60e1a564-328e-4da3-8c52-0f675b68c8f4\" (UID: \"60e1a564-328e-4da3-8c52-0f675b68c8f4\") "
Jul 28 19:36:49.723268 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.723116    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2c4baf3-35e2-4f2d-b844-d33e870f1df3-plugins-config-volume\") pod \"f2c4baf3-35e2-4f2d-b844-d33e870f1df3\" (UID: \"f2c4baf3-35e2-4f2d-b844-d33e870f1df3\") "
Jul 28 19:36:49.723268 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.723137    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vp5p\" (UniqueName: \"kubernetes.io/projected/c68e901c-da46-493e-bbc3-1f68d6571616-kube-api-access-7vp5p\") pod \"c68e901c-da46-493e-bbc3-1f68d6571616\" (UID: \"c68e901c-da46-493e-bbc3-1f68d6571616\") "
Jul 28 19:36:49.723268 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.723257    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c68e901c-da46-493e-bbc3-1f68d6571616-plugins-config-volume\") pod \"c68e901c-da46-493e-bbc3-1f68d6571616\" (UID: \"c68e901c-da46-493e-bbc3-1f68d6571616\") "
Jul 28 19:36:49.723407 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.723332    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60e1a564-328e-4da3-8c52-0f675b68c8f4-openshift-service-ca\") pod \"60e1a564-328e-4da3-8c52-0f675b68c8f4\" (UID: \"60e1a564-328e-4da3-8c52-0f675b68c8f4\") "
Jul 28 19:36:49.723407 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.723397    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjsq9\" (UniqueName: \"kubernetes.io/projected/f2c4baf3-35e2-4f2d-b844-d33e870f1df3-kube-api-access-mjsq9\") pod \"f2c4baf3-35e2-4f2d-b844-d33e870f1df3\" (UID: \"f2c4baf3-35e2-4f2d-b844-d33e870f1df3\") "
Jul 28 19:36:49.723578 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.723555    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4hwq\" (UniqueName: \"kubernetes.io/projected/60e1a564-328e-4da3-8c52-0f675b68c8f4-kube-api-access-m4hwq\") pod \"60e1a564-328e-4da3-8c52-0f675b68c8f4\" (UID: \"60e1a564-328e-4da3-8c52-0f675b68c8f4\") "
Jul 28 19:36:49.723644 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.723578    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2c4baf3-35e2-4f2d-b844-d33e870f1df3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f2c4baf3-35e2-4f2d-b844-d33e870f1df3" (UID: "f2c4baf3-35e2-4f2d-b844-d33e870f1df3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:36:49.723706 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.723653    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c68e901c-da46-493e-bbc3-1f68d6571616-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c68e901c-da46-493e-bbc3-1f68d6571616" (UID: "c68e901c-da46-493e-bbc3-1f68d6571616"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:36:49.723934 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.723909    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c68e901c-da46-493e-bbc3-1f68d6571616-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:36:49.723934 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.723936    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2c4baf3-35e2-4f2d-b844-d33e870f1df3-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:36:49.724128 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.724000    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e1a564-328e-4da3-8c52-0f675b68c8f4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "60e1a564-328e-4da3-8c52-0f675b68c8f4" (UID: "60e1a564-328e-4da3-8c52-0f675b68c8f4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:36:49.725403 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.725380    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c68e901c-da46-493e-bbc3-1f68d6571616-kube-api-access-7vp5p" (OuterVolumeSpecName: "kube-api-access-7vp5p") pod "c68e901c-da46-493e-bbc3-1f68d6571616" (UID: "c68e901c-da46-493e-bbc3-1f68d6571616"). InnerVolumeSpecName "kube-api-access-7vp5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:36:49.725530 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.725399    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e1a564-328e-4da3-8c52-0f675b68c8f4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "60e1a564-328e-4da3-8c52-0f675b68c8f4" (UID: "60e1a564-328e-4da3-8c52-0f675b68c8f4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:36:49.725530 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.725452    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2c4baf3-35e2-4f2d-b844-d33e870f1df3-kube-api-access-mjsq9" (OuterVolumeSpecName: "kube-api-access-mjsq9") pod "f2c4baf3-35e2-4f2d-b844-d33e870f1df3" (UID: "f2c4baf3-35e2-4f2d-b844-d33e870f1df3"). InnerVolumeSpecName "kube-api-access-mjsq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:36:49.725625 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.725583    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60e1a564-328e-4da3-8c52-0f675b68c8f4-kube-api-access-m4hwq" (OuterVolumeSpecName: "kube-api-access-m4hwq") pod "60e1a564-328e-4da3-8c52-0f675b68c8f4" (UID: "60e1a564-328e-4da3-8c52-0f675b68c8f4"). InnerVolumeSpecName "kube-api-access-m4hwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:36:49.824789 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.824719    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjsq9\" (UniqueName: \"kubernetes.io/projected/f2c4baf3-35e2-4f2d-b844-d33e870f1df3-kube-api-access-mjsq9\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:36:49.824789 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.824742    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4hwq\" (UniqueName: \"kubernetes.io/projected/60e1a564-328e-4da3-8c52-0f675b68c8f4-kube-api-access-m4hwq\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:36:49.824789 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.824753    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60e1a564-328e-4da3-8c52-0f675b68c8f4-maas-api-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:36:49.824789 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.824761    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vp5p\" (UniqueName: \"kubernetes.io/projected/c68e901c-da46-493e-bbc3-1f68d6571616-kube-api-access-7vp5p\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:36:49.824789 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:49.824769    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60e1a564-328e-4da3-8c52-0f675b68c8f4-openshift-service-ca\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:36:50.224122 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.224086    2561 generic.go:356] "Generic (PLEG): container finished" podID="f2c4baf3-35e2-4f2d-b844-d33e870f1df3" containerID="38f50a613785a9904a76cfc9ce9e425954624a86ff9e37d4e4067dab4f7f4fda" exitCode=0
Jul 28 19:36:50.224300 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.224132    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"
Jul 28 19:36:50.224300 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.224180    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm" event={"ID":"f2c4baf3-35e2-4f2d-b844-d33e870f1df3","Type":"ContainerDied","Data":"38f50a613785a9904a76cfc9ce9e425954624a86ff9e37d4e4067dab4f7f4fda"}
Jul 28 19:36:50.224300 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.224236    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm" event={"ID":"f2c4baf3-35e2-4f2d-b844-d33e870f1df3","Type":"ContainerDied","Data":"8c6b79c748b8d91c625b72340540a6a1ce1f93cafc4bf5925c3d0bd2de944c6a"}
Jul 28 19:36:50.224300 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.224269    2561 scope.go:117] "RemoveContainer" containerID="38f50a613785a9904a76cfc9ce9e425954624a86ff9e37d4e4067dab4f7f4fda"
Jul 28 19:36:50.225578 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.225560    2561 generic.go:356] "Generic (PLEG): container finished" podID="c68e901c-da46-493e-bbc3-1f68d6571616" containerID="fc7a553bac2431215fbe0911d7f75f1a4c494b32bd83e9fe504daa029804c5a0" exitCode=0
Jul 28 19:36:50.225702 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.225595    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"
Jul 28 19:36:50.225702 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.225640    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm" event={"ID":"c68e901c-da46-493e-bbc3-1f68d6571616","Type":"ContainerDied","Data":"fc7a553bac2431215fbe0911d7f75f1a4c494b32bd83e9fe504daa029804c5a0"}
Jul 28 19:36:50.225804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.225704    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm" event={"ID":"c68e901c-da46-493e-bbc3-1f68d6571616","Type":"ContainerDied","Data":"0790cad8437e0ca3495eed531f6f8c96dcbb5cf259041e963f51c243f9d1d729"}
Jul 28 19:36:50.227068 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.226991    2561 generic.go:356] "Generic (PLEG): container finished" podID="60e1a564-328e-4da3-8c52-0f675b68c8f4" containerID="6a15683e660b27cd710db43704010edbfb790a22ba8f5367dcce1f9ac7c05f4a" exitCode=0
Jul 28 19:36:50.227068 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.227021    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8" event={"ID":"60e1a564-328e-4da3-8c52-0f675b68c8f4","Type":"ContainerDied","Data":"6a15683e660b27cd710db43704010edbfb790a22ba8f5367dcce1f9ac7c05f4a"}
Jul 28 19:36:50.227068 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.227040    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"
Jul 28 19:36:50.227296 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.227089    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8" event={"ID":"60e1a564-328e-4da3-8c52-0f675b68c8f4","Type":"ContainerDied","Data":"315576e35eb0edf166b0086ac8fc747be320b1863103e5b9bcf2102f09bf23f3"}
Jul 28 19:36:50.234890 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.234697    2561 scope.go:117] "RemoveContainer" containerID="38f50a613785a9904a76cfc9ce9e425954624a86ff9e37d4e4067dab4f7f4fda"
Jul 28 19:36:50.234989 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:36:50.234955    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38f50a613785a9904a76cfc9ce9e425954624a86ff9e37d4e4067dab4f7f4fda\": container with ID starting with 38f50a613785a9904a76cfc9ce9e425954624a86ff9e37d4e4067dab4f7f4fda not found: ID does not exist" containerID="38f50a613785a9904a76cfc9ce9e425954624a86ff9e37d4e4067dab4f7f4fda"
Jul 28 19:36:50.235050 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.234995    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38f50a613785a9904a76cfc9ce9e425954624a86ff9e37d4e4067dab4f7f4fda"} err="failed to get container status \"38f50a613785a9904a76cfc9ce9e425954624a86ff9e37d4e4067dab4f7f4fda\": rpc error: code = NotFound desc = could not find container \"38f50a613785a9904a76cfc9ce9e425954624a86ff9e37d4e4067dab4f7f4fda\": container with ID starting with 38f50a613785a9904a76cfc9ce9e425954624a86ff9e37d4e4067dab4f7f4fda not found: ID does not exist"
Jul 28 19:36:50.235050 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.235015    2561 scope.go:117] "RemoveContainer" containerID="fc7a553bac2431215fbe0911d7f75f1a4c494b32bd83e9fe504daa029804c5a0"
Jul 28 19:36:50.243113 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.243092    2561 scope.go:117] "RemoveContainer" containerID="fc7a553bac2431215fbe0911d7f75f1a4c494b32bd83e9fe504daa029804c5a0"
Jul 28 19:36:50.243523 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:36:50.243503    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc7a553bac2431215fbe0911d7f75f1a4c494b32bd83e9fe504daa029804c5a0\": container with ID starting with fc7a553bac2431215fbe0911d7f75f1a4c494b32bd83e9fe504daa029804c5a0 not found: ID does not exist" containerID="fc7a553bac2431215fbe0911d7f75f1a4c494b32bd83e9fe504daa029804c5a0"
Jul 28 19:36:50.243601 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.243529    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc7a553bac2431215fbe0911d7f75f1a4c494b32bd83e9fe504daa029804c5a0"} err="failed to get container status \"fc7a553bac2431215fbe0911d7f75f1a4c494b32bd83e9fe504daa029804c5a0\": rpc error: code = NotFound desc = could not find container \"fc7a553bac2431215fbe0911d7f75f1a4c494b32bd83e9fe504daa029804c5a0\": container with ID starting with fc7a553bac2431215fbe0911d7f75f1a4c494b32bd83e9fe504daa029804c5a0 not found: ID does not exist"
Jul 28 19:36:50.243601 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.243544    2561 scope.go:117] "RemoveContainer" containerID="6a15683e660b27cd710db43704010edbfb790a22ba8f5367dcce1f9ac7c05f4a"
Jul 28 19:36:50.250974 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.250953    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"]
Jul 28 19:36:50.251799 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.251773    2561 scope.go:117] "RemoveContainer" containerID="6a15683e660b27cd710db43704010edbfb790a22ba8f5367dcce1f9ac7c05f4a"
Jul 28 19:36:50.252145 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:36:50.252080    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a15683e660b27cd710db43704010edbfb790a22ba8f5367dcce1f9ac7c05f4a\": container with ID starting with 6a15683e660b27cd710db43704010edbfb790a22ba8f5367dcce1f9ac7c05f4a not found: ID does not exist" containerID="6a15683e660b27cd710db43704010edbfb790a22ba8f5367dcce1f9ac7c05f4a"
Jul 28 19:36:50.252251 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.252154    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a15683e660b27cd710db43704010edbfb790a22ba8f5367dcce1f9ac7c05f4a"} err="failed to get container status \"6a15683e660b27cd710db43704010edbfb790a22ba8f5367dcce1f9ac7c05f4a\": rpc error: code = NotFound desc = could not find container \"6a15683e660b27cd710db43704010edbfb790a22ba8f5367dcce1f9ac7c05f4a\": container with ID starting with 6a15683e660b27cd710db43704010edbfb790a22ba8f5367dcce1f9ac7c05f4a not found: ID does not exist"
Jul 28 19:36:50.254278 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.254259    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b965716e-7c4f786f6b-gqlgm"]
Jul 28 19:36:50.272313 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.272292    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"]
Jul 28 19:36:50.280812 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.280793    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b965716e-664fd589cd-f7bx8"]
Jul 28 19:36:50.322591 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.322562    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"]
Jul 28 19:36:50.339116 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.339087    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b965716e-9bb75dbd4-v4jfm"]
Jul 28 19:36:50.604877 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.604798    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60e1a564-328e-4da3-8c52-0f675b68c8f4" path="/var/lib/kubelet/pods/60e1a564-328e-4da3-8c52-0f675b68c8f4/volumes"
Jul 28 19:36:50.605216 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.605200    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c68e901c-da46-493e-bbc3-1f68d6571616" path="/var/lib/kubelet/pods/c68e901c-da46-493e-bbc3-1f68d6571616/volumes"
Jul 28 19:36:50.605553 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:36:50.605540    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2c4baf3-35e2-4f2d-b844-d33e870f1df3" path="/var/lib/kubelet/pods/f2c4baf3-35e2-4f2d-b844-d33e870f1df3/volumes"
Jul 28 19:37:32.582415 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:37:32.582384    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:37:32.588389 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:37:32.588366    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:37:32.588738 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:37:32.588721    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:37:32.600974 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:37:32.600938    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:38:44.398742 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.398710    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"]
Jul 28 19:38:44.399264 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.399244    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2c4baf3-35e2-4f2d-b844-d33e870f1df3" containerName="payload-processing"
Jul 28 19:38:44.399311 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.399269    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2c4baf3-35e2-4f2d-b844-d33e870f1df3" containerName="payload-processing"
Jul 28 19:38:44.399311 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.399285    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c68e901c-da46-493e-bbc3-1f68d6571616" containerName="payload-pre-processing"
Jul 28 19:38:44.399311 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.399295    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c68e901c-da46-493e-bbc3-1f68d6571616" containerName="payload-pre-processing"
Jul 28 19:38:44.399407 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.399313    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e1a564-328e-4da3-8c52-0f675b68c8f4" containerName="maas-api"
Jul 28 19:38:44.399407 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.399322    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e1a564-328e-4da3-8c52-0f675b68c8f4" containerName="maas-api"
Jul 28 19:38:44.399495 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.399420    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2c4baf3-35e2-4f2d-b844-d33e870f1df3" containerName="payload-processing"
Jul 28 19:38:44.399495 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.399434    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c68e901c-da46-493e-bbc3-1f68d6571616" containerName="payload-pre-processing"
Jul 28 19:38:44.399495 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.399442    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="60e1a564-328e-4da3-8c52-0f675b68c8f4" containerName="maas-api"
Jul 28 19:38:44.402876 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.402858    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:38:44.405802 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.405772    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:38:44.406599 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.406573    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 19:38:44.406715 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.406601    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z259w\""
Jul 28 19:38:44.406715 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.406668    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-e55e7d-serving-cert\""
Jul 28 19:38:44.411922 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.411899    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"]
Jul 28 19:38:44.440960 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.440932    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce69b884-f0d2-4571-9438-ecba796aa12c-maas-api-tls\") pod \"maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k\" (UID: \"ce69b884-f0d2-4571-9438-ecba796aa12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:38:44.441059 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.441023    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce69b884-f0d2-4571-9438-ecba796aa12c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k\" (UID: \"ce69b884-f0d2-4571-9438-ecba796aa12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:38:44.441059 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.441040    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfsm7\" (UniqueName: \"kubernetes.io/projected/ce69b884-f0d2-4571-9438-ecba796aa12c-kube-api-access-vfsm7\") pod \"maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k\" (UID: \"ce69b884-f0d2-4571-9438-ecba796aa12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:38:44.542199 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.542169    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce69b884-f0d2-4571-9438-ecba796aa12c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k\" (UID: \"ce69b884-f0d2-4571-9438-ecba796aa12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:38:44.542199 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.542200    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfsm7\" (UniqueName: \"kubernetes.io/projected/ce69b884-f0d2-4571-9438-ecba796aa12c-kube-api-access-vfsm7\") pod \"maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k\" (UID: \"ce69b884-f0d2-4571-9438-ecba796aa12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:38:44.542381 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.542270    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce69b884-f0d2-4571-9438-ecba796aa12c-maas-api-tls\") pod \"maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k\" (UID: \"ce69b884-f0d2-4571-9438-ecba796aa12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:38:44.542419 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:38:44.542403    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-serving-cert: secret "maas-api-e2e-inf-a-e55e7d-serving-cert" not found
Jul 28 19:38:44.542533 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:38:44.542520    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce69b884-f0d2-4571-9438-ecba796aa12c-maas-api-tls podName:ce69b884-f0d2-4571-9438-ecba796aa12c nodeName:}" failed. No retries permitted until 2026-07-28 19:38:45.042469072 +0000 UTC m=+2172.988953919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ce69b884-f0d2-4571-9438-ecba796aa12c-maas-api-tls") pod "maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k" (UID: "ce69b884-f0d2-4571-9438-ecba796aa12c") : secret "maas-api-e2e-inf-a-e55e7d-serving-cert" not found
Jul 28 19:38:44.542863 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.542848    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce69b884-f0d2-4571-9438-ecba796aa12c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k\" (UID: \"ce69b884-f0d2-4571-9438-ecba796aa12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:38:44.550358 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:44.550338    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfsm7\" (UniqueName: \"kubernetes.io/projected/ce69b884-f0d2-4571-9438-ecba796aa12c-kube-api-access-vfsm7\") pod \"maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k\" (UID: \"ce69b884-f0d2-4571-9438-ecba796aa12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:38:45.045746 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.045704    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce69b884-f0d2-4571-9438-ecba796aa12c-maas-api-tls\") pod \"maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k\" (UID: \"ce69b884-f0d2-4571-9438-ecba796aa12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:38:45.045951 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:38:45.045853    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-serving-cert: secret "maas-api-e2e-inf-a-e55e7d-serving-cert" not found
Jul 28 19:38:45.045951 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:38:45.045924    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce69b884-f0d2-4571-9438-ecba796aa12c-maas-api-tls podName:ce69b884-f0d2-4571-9438-ecba796aa12c nodeName:}" failed. No retries permitted until 2026-07-28 19:38:46.045906065 +0000 UTC m=+2173.992390915 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ce69b884-f0d2-4571-9438-ecba796aa12c-maas-api-tls") pod "maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k" (UID: "ce69b884-f0d2-4571-9438-ecba796aa12c") : secret "maas-api-e2e-inf-a-e55e7d-serving-cert" not found
Jul 28 19:38:45.197542 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.197499    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"]
Jul 28 19:38:45.201350 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.201325    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"
Jul 28 19:38:45.203658 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.203631    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-e55e7d\""
Jul 28 19:38:45.203658 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.203651    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-e55e7d-dockercfg-sbnfm\""
Jul 28 19:38:45.211130 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.211103    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"]
Jul 28 19:38:45.234099 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.234063    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"]
Jul 28 19:38:45.237300 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.237284    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"
Jul 28 19:38:45.246629 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.246601    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"]
Jul 28 19:38:45.247008 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.246987    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40708c57-108b-4f97-9dc5-c67e45c09ce9-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd\" (UID: \"40708c57-108b-4f97-9dc5-c67e45c09ce9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"
Jul 28 19:38:45.247100 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.247054    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9shgd\" (UniqueName: \"kubernetes.io/projected/40708c57-108b-4f97-9dc5-c67e45c09ce9-kube-api-access-9shgd\") pod \"payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd\" (UID: \"40708c57-108b-4f97-9dc5-c67e45c09ce9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"
Jul 28 19:38:45.348068 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.347986    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0973213-7f08-4896-9d60-1252e654a06d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494\" (UID: \"d0973213-7f08-4896-9d60-1252e654a06d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"
Jul 28 19:38:45.348068 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.348057    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40708c57-108b-4f97-9dc5-c67e45c09ce9-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd\" (UID: \"40708c57-108b-4f97-9dc5-c67e45c09ce9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"
Jul 28 19:38:45.348287 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.348109    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9shgd\" (UniqueName: \"kubernetes.io/projected/40708c57-108b-4f97-9dc5-c67e45c09ce9-kube-api-access-9shgd\") pod \"payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd\" (UID: \"40708c57-108b-4f97-9dc5-c67e45c09ce9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"
Jul 28 19:38:45.348287 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.348148    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzv25\" (UniqueName: \"kubernetes.io/projected/d0973213-7f08-4896-9d60-1252e654a06d-kube-api-access-mzv25\") pod \"payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494\" (UID: \"d0973213-7f08-4896-9d60-1252e654a06d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"
Jul 28 19:38:45.348687 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.348667    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40708c57-108b-4f97-9dc5-c67e45c09ce9-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd\" (UID: \"40708c57-108b-4f97-9dc5-c67e45c09ce9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"
Jul 28 19:38:45.355835 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.355809    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9shgd\" (UniqueName: \"kubernetes.io/projected/40708c57-108b-4f97-9dc5-c67e45c09ce9-kube-api-access-9shgd\") pod \"payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd\" (UID: \"40708c57-108b-4f97-9dc5-c67e45c09ce9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"
Jul 28 19:38:45.449254 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.449221    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzv25\" (UniqueName: \"kubernetes.io/projected/d0973213-7f08-4896-9d60-1252e654a06d-kube-api-access-mzv25\") pod \"payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494\" (UID: \"d0973213-7f08-4896-9d60-1252e654a06d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"
Jul 28 19:38:45.449621 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.449261    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0973213-7f08-4896-9d60-1252e654a06d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494\" (UID: \"d0973213-7f08-4896-9d60-1252e654a06d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"
Jul 28 19:38:45.450006 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.449988    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0973213-7f08-4896-9d60-1252e654a06d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494\" (UID: \"d0973213-7f08-4896-9d60-1252e654a06d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"
Jul 28 19:38:45.456629 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.456609    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzv25\" (UniqueName: \"kubernetes.io/projected/d0973213-7f08-4896-9d60-1252e654a06d-kube-api-access-mzv25\") pod \"payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494\" (UID: \"d0973213-7f08-4896-9d60-1252e654a06d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"
Jul 28 19:38:45.513533 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.513493    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"
Jul 28 19:38:45.549662 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.549634    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"
Jul 28 19:38:45.686285 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.686257    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"]
Jul 28 19:38:45.688289 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:38:45.688263    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0973213_7f08_4896_9d60_1252e654a06d.slice/crio-dbdcd25ca2bd117fc455c315d3705c13d5f6c4164c4b1fd79e5df0470d699edd WatchSource:0}: Error finding container dbdcd25ca2bd117fc455c315d3705c13d5f6c4164c4b1fd79e5df0470d699edd: Status 404 returned error can't find the container with id dbdcd25ca2bd117fc455c315d3705c13d5f6c4164c4b1fd79e5df0470d699edd
Jul 28 19:38:45.751529 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.751497    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494" event={"ID":"d0973213-7f08-4896-9d60-1252e654a06d","Type":"ContainerStarted","Data":"dc771a7905edad2c3a0ef622e2bec24a6b294e61f7cf959252a1be8fc4d9c87e"}
Jul 28 19:38:45.751649 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.751555    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494" event={"ID":"d0973213-7f08-4896-9d60-1252e654a06d","Type":"ContainerStarted","Data":"dbdcd25ca2bd117fc455c315d3705c13d5f6c4164c4b1fd79e5df0470d699edd"}
Jul 28 19:38:45.751728 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.751699    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"
Jul 28 19:38:45.768063 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.768017    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494" podStartSLOduration=0.768002045 podStartE2EDuration="768.002045ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:38:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:38:45.766643923 +0000 UTC m=+2173.713128806" watchObservedRunningTime="2026-07-28 19:38:45.768002045 +0000 UTC m=+2173.714486911"
Jul 28 19:38:45.866865 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:45.866796    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"]
Jul 28 19:38:45.869516 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:38:45.869459    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40708c57_108b_4f97_9dc5_c67e45c09ce9.slice/crio-03ec6b56a79d7352bb6832fc6caf0619d6afd8eab2503f562fe02d13079fb1ce WatchSource:0}: Error finding container 03ec6b56a79d7352bb6832fc6caf0619d6afd8eab2503f562fe02d13079fb1ce: Status 404 returned error can't find the container with id 03ec6b56a79d7352bb6832fc6caf0619d6afd8eab2503f562fe02d13079fb1ce
Jul 28 19:38:46.054065 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:46.054033    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce69b884-f0d2-4571-9438-ecba796aa12c-maas-api-tls\") pod \"maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k\" (UID: \"ce69b884-f0d2-4571-9438-ecba796aa12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:38:46.056193 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:46.056169    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce69b884-f0d2-4571-9438-ecba796aa12c-maas-api-tls\") pod \"maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k\" (UID: \"ce69b884-f0d2-4571-9438-ecba796aa12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:38:46.215031 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:46.214997    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:38:46.351556 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:46.351525    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"]
Jul 28 19:38:46.354537 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:38:46.354473    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce69b884_f0d2_4571_9438_ecba796aa12c.slice/crio-9531ad457586c34c0bcb32977df3f0c18e5d8e769aae27579d7a464a524be01c WatchSource:0}: Error finding container 9531ad457586c34c0bcb32977df3f0c18e5d8e769aae27579d7a464a524be01c: Status 404 returned error can't find the container with id 9531ad457586c34c0bcb32977df3f0c18e5d8e769aae27579d7a464a524be01c
Jul 28 19:38:46.756365 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:46.756335    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k" event={"ID":"ce69b884-f0d2-4571-9438-ecba796aa12c","Type":"ContainerStarted","Data":"9531ad457586c34c0bcb32977df3f0c18e5d8e769aae27579d7a464a524be01c"}
Jul 28 19:38:46.757796 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:46.757767    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd" event={"ID":"40708c57-108b-4f97-9dc5-c67e45c09ce9","Type":"ContainerStarted","Data":"33c23d26a8c3491a3c42d393d355a451013dd3b687e5ac82b237ade6f939d779"}
Jul 28 19:38:46.757904 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:46.757821    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd" event={"ID":"40708c57-108b-4f97-9dc5-c67e45c09ce9","Type":"ContainerStarted","Data":"03ec6b56a79d7352bb6832fc6caf0619d6afd8eab2503f562fe02d13079fb1ce"}
Jul 28 19:38:46.773958 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:46.773911    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd" podStartSLOduration=1.773895802 podStartE2EDuration="1.773895802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:38:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:38:46.772404758 +0000 UTC m=+2174.718889629" watchObservedRunningTime="2026-07-28 19:38:46.773895802 +0000 UTC m=+2174.720380670"
Jul 28 19:38:47.763575 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:47.763534    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k" event={"ID":"ce69b884-f0d2-4571-9438-ecba796aa12c","Type":"ContainerStarted","Data":"9b1f232dd8c7de230cf3faee177d3cc82ed430675f75dc10ef91f56e87c1c1c9"}
Jul 28 19:38:47.763994 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:47.763665    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"
Jul 28 19:38:47.780741 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:47.780687    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k" podStartSLOduration=3.264688651 podStartE2EDuration="3.780668527s" totalImagesPullingTime="515.979876ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:38:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:38:47.778651272 +0000 UTC m=+2175.725136141" watchObservedRunningTime="2026-07-28 19:38:47.780668527 +0000 UTC m=+2175.727153393"
Jul 28 19:38:48.767833 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:48.767795    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:38:54.778394 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:54.778360    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:38:56.760688 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:56.760658    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"
Jul 28 19:38:58.769698 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:38:58.769664    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"
Jul 28 19:39:12.465685 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:12.465651    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"]
Jul 28 19:39:12.469301 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:12.469276    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"
Jul 28 19:39:12.471616 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:12.471593    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-5f6911-serving-cert\""
Jul 28 19:39:12.480370 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:12.480347    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"]
Jul 28 19:39:12.594679 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:12.594652    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4mr8\" (UniqueName: \"kubernetes.io/projected/23d55773-35ce-43c7-a817-b47f98b8adc9-kube-api-access-v4mr8\") pod \"maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b\" (UID: \"23d55773-35ce-43c7-a817-b47f98b8adc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"
Jul 28 19:39:12.594797 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:12.594712    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23d55773-35ce-43c7-a817-b47f98b8adc9-maas-api-tls\") pod \"maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b\" (UID: \"23d55773-35ce-43c7-a817-b47f98b8adc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"
Jul 28 19:39:12.594797 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:12.594765    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23d55773-35ce-43c7-a817-b47f98b8adc9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b\" (UID: \"23d55773-35ce-43c7-a817-b47f98b8adc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"
Jul 28 19:39:12.696175 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:12.696151    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4mr8\" (UniqueName: \"kubernetes.io/projected/23d55773-35ce-43c7-a817-b47f98b8adc9-kube-api-access-v4mr8\") pod \"maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b\" (UID: \"23d55773-35ce-43c7-a817-b47f98b8adc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"
Jul 28 19:39:12.696299 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:12.696214    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23d55773-35ce-43c7-a817-b47f98b8adc9-maas-api-tls\") pod \"maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b\" (UID: \"23d55773-35ce-43c7-a817-b47f98b8adc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"
Jul 28 19:39:12.696299 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:12.696240    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23d55773-35ce-43c7-a817-b47f98b8adc9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b\" (UID: \"23d55773-35ce-43c7-a817-b47f98b8adc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"
Jul 28 19:39:12.696887 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:12.696863    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23d55773-35ce-43c7-a817-b47f98b8adc9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b\" (UID: \"23d55773-35ce-43c7-a817-b47f98b8adc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"
Jul 28 19:39:12.698674 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:12.698654    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23d55773-35ce-43c7-a817-b47f98b8adc9-maas-api-tls\") pod \"maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b\" (UID: \"23d55773-35ce-43c7-a817-b47f98b8adc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"
Jul 28 19:39:12.705035 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:12.705011    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4mr8\" (UniqueName: \"kubernetes.io/projected/23d55773-35ce-43c7-a817-b47f98b8adc9-kube-api-access-v4mr8\") pod \"maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b\" (UID: \"23d55773-35ce-43c7-a817-b47f98b8adc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"
Jul 28 19:39:12.782838 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:12.782786    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"
Jul 28 19:39:12.948313 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:12.948285    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"]
Jul 28 19:39:12.950461 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:39:12.950435    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23d55773_35ce_43c7_a817_b47f98b8adc9.slice/crio-3c58623c826914ed12487edc4169d9749fc8586c9417d4d54c3cfeaa0caf03e0 WatchSource:0}: Error finding container 3c58623c826914ed12487edc4169d9749fc8586c9417d4d54c3cfeaa0caf03e0: Status 404 returned error can't find the container with id 3c58623c826914ed12487edc4169d9749fc8586c9417d4d54c3cfeaa0caf03e0
Jul 28 19:39:13.170111 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.170041    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"]
Jul 28 19:39:13.174710 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.174691    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"
Jul 28 19:39:13.177549 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.177533    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-5f6911\""
Jul 28 19:39:13.177633 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.177579    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-5f6911-dockercfg-m9qsk\""
Jul 28 19:39:13.188457 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.188437    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"]
Jul 28 19:39:13.200901 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.200879    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"]
Jul 28 19:39:13.201560 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.201539    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9csn6\" (UniqueName: \"kubernetes.io/projected/24b484e6-5c27-4bab-9bed-029ded6b2e4e-kube-api-access-9csn6\") pod \"payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx\" (UID: \"24b484e6-5c27-4bab-9bed-029ded6b2e4e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"
Jul 28 19:39:13.201653 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.201617    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24b484e6-5c27-4bab-9bed-029ded6b2e4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx\" (UID: \"24b484e6-5c27-4bab-9bed-029ded6b2e4e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"
Jul 28 19:39:13.204413 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.204396    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"
Jul 28 19:39:13.214210 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.214189    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"]
Jul 28 19:39:13.302219 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.302198    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9csn6\" (UniqueName: \"kubernetes.io/projected/24b484e6-5c27-4bab-9bed-029ded6b2e4e-kube-api-access-9csn6\") pod \"payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx\" (UID: \"24b484e6-5c27-4bab-9bed-029ded6b2e4e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"
Jul 28 19:39:13.302321 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.302249    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn5sg\" (UniqueName: \"kubernetes.io/projected/e571dbc9-da84-4715-a7df-8c18a00cda90-kube-api-access-jn5sg\") pod \"payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf\" (UID: \"e571dbc9-da84-4715-a7df-8c18a00cda90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"
Jul 28 19:39:13.302321 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.302276    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e571dbc9-da84-4715-a7df-8c18a00cda90-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf\" (UID: \"e571dbc9-da84-4715-a7df-8c18a00cda90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"
Jul 28 19:39:13.302416 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.302321    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24b484e6-5c27-4bab-9bed-029ded6b2e4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx\" (UID: \"24b484e6-5c27-4bab-9bed-029ded6b2e4e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"
Jul 28 19:39:13.303050 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.303029    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24b484e6-5c27-4bab-9bed-029ded6b2e4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx\" (UID: \"24b484e6-5c27-4bab-9bed-029ded6b2e4e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"
Jul 28 19:39:13.313926 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.313907    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9csn6\" (UniqueName: \"kubernetes.io/projected/24b484e6-5c27-4bab-9bed-029ded6b2e4e-kube-api-access-9csn6\") pod \"payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx\" (UID: \"24b484e6-5c27-4bab-9bed-029ded6b2e4e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"
Jul 28 19:39:13.403230 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.403207    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn5sg\" (UniqueName: \"kubernetes.io/projected/e571dbc9-da84-4715-a7df-8c18a00cda90-kube-api-access-jn5sg\") pod \"payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf\" (UID: \"e571dbc9-da84-4715-a7df-8c18a00cda90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"
Jul 28 19:39:13.403352 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.403244    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e571dbc9-da84-4715-a7df-8c18a00cda90-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf\" (UID: \"e571dbc9-da84-4715-a7df-8c18a00cda90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"
Jul 28 19:39:13.404012 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.403995    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e571dbc9-da84-4715-a7df-8c18a00cda90-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf\" (UID: \"e571dbc9-da84-4715-a7df-8c18a00cda90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"
Jul 28 19:39:13.412955 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.412932    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn5sg\" (UniqueName: \"kubernetes.io/projected/e571dbc9-da84-4715-a7df-8c18a00cda90-kube-api-access-jn5sg\") pod \"payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf\" (UID: \"e571dbc9-da84-4715-a7df-8c18a00cda90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"
Jul 28 19:39:13.485232 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.485208    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"
Jul 28 19:39:13.516345 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.516320    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"
Jul 28 19:39:13.627272 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.627244    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"]
Jul 28 19:39:13.630234 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:39:13.630208    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b484e6_5c27_4bab_9bed_029ded6b2e4e.slice/crio-95aaf157885000d4b23af4c42f124615d637dfde2908fcf54784354290b819c0 WatchSource:0}: Error finding container 95aaf157885000d4b23af4c42f124615d637dfde2908fcf54784354290b819c0: Status 404 returned error can't find the container with id 95aaf157885000d4b23af4c42f124615d637dfde2908fcf54784354290b819c0
Jul 28 19:39:13.659602 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.659575    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"]
Jul 28 19:39:13.662464 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:39:13.662440    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode571dbc9_da84_4715_a7df_8c18a00cda90.slice/crio-aea18e7849b68a83f70c8088730a57bf47b5cd9e2fcc08c39d329e656726a873 WatchSource:0}: Error finding container aea18e7849b68a83f70c8088730a57bf47b5cd9e2fcc08c39d329e656726a873: Status 404 returned error can't find the container with id aea18e7849b68a83f70c8088730a57bf47b5cd9e2fcc08c39d329e656726a873
Jul 28 19:39:13.890446 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.890359    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf" event={"ID":"e571dbc9-da84-4715-a7df-8c18a00cda90","Type":"ContainerStarted","Data":"57d425cb92edd9ace1a3e2d65691b8eb0e35b737feb7b2afafe6b93f00d32e73"}
Jul 28 19:39:13.890446 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.890423    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf" event={"ID":"e571dbc9-da84-4715-a7df-8c18a00cda90","Type":"ContainerStarted","Data":"aea18e7849b68a83f70c8088730a57bf47b5cd9e2fcc08c39d329e656726a873"}
Jul 28 19:39:13.890676 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.890450    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"
Jul 28 19:39:13.891890 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.891859    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx" event={"ID":"24b484e6-5c27-4bab-9bed-029ded6b2e4e","Type":"ContainerStarted","Data":"b223cf0526c7718c1a42e05274954ba2ef40824aa2e4871e84b053863295bce7"}
Jul 28 19:39:13.892024 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.891905    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"
Jul 28 19:39:13.892024 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.891926    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx" event={"ID":"24b484e6-5c27-4bab-9bed-029ded6b2e4e","Type":"ContainerStarted","Data":"95aaf157885000d4b23af4c42f124615d637dfde2908fcf54784354290b819c0"}
Jul 28 19:39:13.893421 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.893397    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b" event={"ID":"23d55773-35ce-43c7-a817-b47f98b8adc9","Type":"ContainerStarted","Data":"540d0b16535fbb05553ad9cfffa21e162c5bce943af7a34b9ab845927cfdb5f5"}
Jul 28 19:39:13.893564 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.893438    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b" event={"ID":"23d55773-35ce-43c7-a817-b47f98b8adc9","Type":"ContainerStarted","Data":"3c58623c826914ed12487edc4169d9749fc8586c9417d4d54c3cfeaa0caf03e0"}
Jul 28 19:39:13.893564 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.893539    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"
Jul 28 19:39:13.906898 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.906849    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf" podStartSLOduration=0.906831573 podStartE2EDuration="906.831573ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:39:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:39:13.904991458 +0000 UTC m=+2201.851476351" watchObservedRunningTime="2026-07-28 19:39:13.906831573 +0000 UTC m=+2201.853316440"
Jul 28 19:39:13.920389 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.920345    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b" podStartSLOduration=1.598146334 podStartE2EDuration="1.920330079s" totalImagesPullingTime="322.183745ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:39:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:39:13.919839008 +0000 UTC m=+2201.866323901" watchObservedRunningTime="2026-07-28 19:39:13.920330079 +0000 UTC m=+2201.866814970"
Jul 28 19:39:13.936530 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:13.936476    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx" podStartSLOduration=0.936463241 podStartE2EDuration="936.463241ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:39:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:39:13.935544117 +0000 UTC m=+2201.882029014" watchObservedRunningTime="2026-07-28 19:39:13.936463241 +0000 UTC m=+2201.882948104"
Jul 28 19:39:19.904718 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:19.904644    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"
Jul 28 19:39:24.900131 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:24.900097    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"
Jul 28 19:39:24.900680 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:24.900312    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"
Jul 28 19:39:55.939694 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:55.939601    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"]
Jul 28 19:39:55.943647 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:55.943623    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:55.946090 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:55.946068    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e55e7d\"/\"openshift-service-ca.crt\""
Jul 28 19:39:55.947099 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:55.947072    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e55e7d\"/\"kube-root-ca.crt\""
Jul 28 19:39:55.947842 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:55.947662    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e55e7d\"/\"default-dockercfg-77kvr\""
Jul 28 19:39:55.947994 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:55.947968    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e55e7d\"/\"test-model-e55e7d-kserve-self-signed-certs\""
Jul 28 19:39:55.950836 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:55.950809    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"]
Jul 28 19:39:56.078797 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.078765    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-home\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.078961 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.078823    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-dshm\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.078961 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.078920    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99460cfb-2fae-4c23-b93f-2b782944b44c-tls-certs\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.079067 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.078960    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvq7k\" (UniqueName: \"kubernetes.io/projected/99460cfb-2fae-4c23-b93f-2b782944b44c-kube-api-access-pvq7k\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.079109 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.079076    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-model-cache\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.079156 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.079127    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-tmp-dir\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.180457 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.180425    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-dshm\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.180617 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.180504    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99460cfb-2fae-4c23-b93f-2b782944b44c-tls-certs\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.180617 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.180543    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvq7k\" (UniqueName: \"kubernetes.io/projected/99460cfb-2fae-4c23-b93f-2b782944b44c-kube-api-access-pvq7k\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.180617 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.180605    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-model-cache\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.180786 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.180658    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-tmp-dir\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.180786 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.180679    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-home\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.180976 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.180957    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-model-cache\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.181072 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.181012    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-home\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.181072 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.181058    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-tmp-dir\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.182795 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.182772    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-dshm\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.183161 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.183145    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99460cfb-2fae-4c23-b93f-2b782944b44c-tls-certs\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.189028 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.189010    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvq7k\" (UniqueName: \"kubernetes.io/projected/99460cfb-2fae-4c23-b93f-2b782944b44c-kube-api-access-pvq7k\") pod \"test-model-e55e7d-kserve-5544d5b4f9-n74z4\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") " pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.256940 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.256919    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:39:56.387081 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:56.387056    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"]
Jul 28 19:39:56.389322 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:39:56.389296    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99460cfb_2fae_4c23_b93f_2b782944b44c.slice/crio-e693fb488c7572fd456294ab19b811c53bf91a1f28adac359ef8e4140df13e03 WatchSource:0}: Error finding container e693fb488c7572fd456294ab19b811c53bf91a1f28adac359ef8e4140df13e03: Status 404 returned error can't find the container with id e693fb488c7572fd456294ab19b811c53bf91a1f28adac359ef8e4140df13e03
Jul 28 19:39:57.124234 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:57.124200    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4" event={"ID":"99460cfb-2fae-4c23-b93f-2b782944b44c","Type":"ContainerStarted","Data":"1bda8158f3f58d53906393f1cca8c0a48c6d35a122799c5c567fddd980222460"}
Jul 28 19:39:57.124645 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:57.124245    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4" event={"ID":"99460cfb-2fae-4c23-b93f-2b782944b44c","Type":"ContainerStarted","Data":"e693fb488c7572fd456294ab19b811c53bf91a1f28adac359ef8e4140df13e03"}
Jul 28 19:39:57.141781 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:39:57.141738    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4" podStartSLOduration=2.141721556 podStartE2EDuration="2.141721556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:39:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:39:57.13972469 +0000 UTC m=+2245.086209560" watchObservedRunningTime="2026-07-28 19:39:57.141721556 +0000 UTC m=+2245.088206425"
Jul 28 19:40:06.257678 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:40:06.257639    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:40:06.257678 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:40:06.257689    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:40:06.270565 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:40:06.270535    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:40:07.183929 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:40:07.183902    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:41:22.325779 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:22.325744    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"]
Jul 28 19:41:34.136100 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.136028    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"]
Jul 28 19:41:34.136546 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.136228    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k" podUID="ce69b884-f0d2-4571-9438-ecba796aa12c" containerName="maas-api" containerID="cri-o://9b1f232dd8c7de230cf3faee177d3cc82ed430675f75dc10ef91f56e87c1c1c9" gracePeriod=30
Jul 28 19:41:34.183778 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.183746    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"]
Jul 28 19:41:34.183998 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.183974    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494" podUID="d0973213-7f08-4896-9d60-1252e654a06d" containerName="payload-processing" containerID="cri-o://dc771a7905edad2c3a0ef622e2bec24a6b294e61f7cf959252a1be8fc4d9c87e" gracePeriod=30
Jul 28 19:41:34.191817 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.191787    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"]
Jul 28 19:41:34.191998 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.191974    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd" podUID="40708c57-108b-4f97-9dc5-c67e45c09ce9" containerName="payload-pre-processing" containerID="cri-o://33c23d26a8c3491a3c42d393d355a451013dd3b687e5ac82b237ade6f939d779" gracePeriod=30
Jul 28 19:41:34.493290 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.493263    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"
Jul 28 19:41:34.496603 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.496575    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"
Jul 28 19:41:34.539118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.539089    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0973213-7f08-4896-9d60-1252e654a06d-plugins-config-volume\") pod \"d0973213-7f08-4896-9d60-1252e654a06d\" (UID: \"d0973213-7f08-4896-9d60-1252e654a06d\") "
Jul 28 19:41:34.539282 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.539128    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9shgd\" (UniqueName: \"kubernetes.io/projected/40708c57-108b-4f97-9dc5-c67e45c09ce9-kube-api-access-9shgd\") pod \"40708c57-108b-4f97-9dc5-c67e45c09ce9\" (UID: \"40708c57-108b-4f97-9dc5-c67e45c09ce9\") "
Jul 28 19:41:34.539282 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.539167    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzv25\" (UniqueName: \"kubernetes.io/projected/d0973213-7f08-4896-9d60-1252e654a06d-kube-api-access-mzv25\") pod \"d0973213-7f08-4896-9d60-1252e654a06d\" (UID: \"d0973213-7f08-4896-9d60-1252e654a06d\") "
Jul 28 19:41:34.539282 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.539227    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40708c57-108b-4f97-9dc5-c67e45c09ce9-plugins-config-volume\") pod \"40708c57-108b-4f97-9dc5-c67e45c09ce9\" (UID: \"40708c57-108b-4f97-9dc5-c67e45c09ce9\") "
Jul 28 19:41:34.539575 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.539546    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0973213-7f08-4896-9d60-1252e654a06d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d0973213-7f08-4896-9d60-1252e654a06d" (UID: "d0973213-7f08-4896-9d60-1252e654a06d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:41:34.539810 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.539776    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40708c57-108b-4f97-9dc5-c67e45c09ce9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "40708c57-108b-4f97-9dc5-c67e45c09ce9" (UID: "40708c57-108b-4f97-9dc5-c67e45c09ce9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:41:34.541800 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.541775    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40708c57-108b-4f97-9dc5-c67e45c09ce9-kube-api-access-9shgd" (OuterVolumeSpecName: "kube-api-access-9shgd") pod "40708c57-108b-4f97-9dc5-c67e45c09ce9" (UID: "40708c57-108b-4f97-9dc5-c67e45c09ce9"). InnerVolumeSpecName "kube-api-access-9shgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:41:34.541888 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.541808    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0973213-7f08-4896-9d60-1252e654a06d-kube-api-access-mzv25" (OuterVolumeSpecName: "kube-api-access-mzv25") pod "d0973213-7f08-4896-9d60-1252e654a06d" (UID: "d0973213-7f08-4896-9d60-1252e654a06d"). InnerVolumeSpecName "kube-api-access-mzv25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:41:34.584825 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.584798    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:41:34.619795 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.619766    2561 generic.go:356] "Generic (PLEG): container finished" podID="40708c57-108b-4f97-9dc5-c67e45c09ce9" containerID="33c23d26a8c3491a3c42d393d355a451013dd3b687e5ac82b237ade6f939d779" exitCode=0
Jul 28 19:41:34.619915 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.619811    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"
Jul 28 19:41:34.619915 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.619852    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd" event={"ID":"40708c57-108b-4f97-9dc5-c67e45c09ce9","Type":"ContainerDied","Data":"33c23d26a8c3491a3c42d393d355a451013dd3b687e5ac82b237ade6f939d779"}
Jul 28 19:41:34.619915 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.619900    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd" event={"ID":"40708c57-108b-4f97-9dc5-c67e45c09ce9","Type":"ContainerDied","Data":"03ec6b56a79d7352bb6832fc6caf0619d6afd8eab2503f562fe02d13079fb1ce"}
Jul 28 19:41:34.620089 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.619930    2561 scope.go:117] "RemoveContainer" containerID="33c23d26a8c3491a3c42d393d355a451013dd3b687e5ac82b237ade6f939d779"
Jul 28 19:41:34.621350 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.621325    2561 generic.go:356] "Generic (PLEG): container finished" podID="ce69b884-f0d2-4571-9438-ecba796aa12c" containerID="9b1f232dd8c7de230cf3faee177d3cc82ed430675f75dc10ef91f56e87c1c1c9" exitCode=0
Jul 28 19:41:34.621460 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.621368    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"
Jul 28 19:41:34.621460 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.621421    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k" event={"ID":"ce69b884-f0d2-4571-9438-ecba796aa12c","Type":"ContainerDied","Data":"9b1f232dd8c7de230cf3faee177d3cc82ed430675f75dc10ef91f56e87c1c1c9"}
Jul 28 19:41:34.621663 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.621465    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k" event={"ID":"ce69b884-f0d2-4571-9438-ecba796aa12c","Type":"ContainerDied","Data":"9531ad457586c34c0bcb32977df3f0c18e5d8e769aae27579d7a464a524be01c"}
Jul 28 19:41:34.622976 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.622953    2561 generic.go:356] "Generic (PLEG): container finished" podID="d0973213-7f08-4896-9d60-1252e654a06d" containerID="dc771a7905edad2c3a0ef622e2bec24a6b294e61f7cf959252a1be8fc4d9c87e" exitCode=0
Jul 28 19:41:34.623079 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.622984    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"
Jul 28 19:41:34.623079 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.622988    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494" event={"ID":"d0973213-7f08-4896-9d60-1252e654a06d","Type":"ContainerDied","Data":"dc771a7905edad2c3a0ef622e2bec24a6b294e61f7cf959252a1be8fc4d9c87e"}
Jul 28 19:41:34.623079 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.623027    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494" event={"ID":"d0973213-7f08-4896-9d60-1252e654a06d","Type":"ContainerDied","Data":"dbdcd25ca2bd117fc455c315d3705c13d5f6c4164c4b1fd79e5df0470d699edd"}
Jul 28 19:41:34.629799 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.629781    2561 scope.go:117] "RemoveContainer" containerID="33c23d26a8c3491a3c42d393d355a451013dd3b687e5ac82b237ade6f939d779"
Jul 28 19:41:34.630098 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:41:34.630080    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33c23d26a8c3491a3c42d393d355a451013dd3b687e5ac82b237ade6f939d779\": container with ID starting with 33c23d26a8c3491a3c42d393d355a451013dd3b687e5ac82b237ade6f939d779 not found: ID does not exist" containerID="33c23d26a8c3491a3c42d393d355a451013dd3b687e5ac82b237ade6f939d779"
Jul 28 19:41:34.630157 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.630106    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33c23d26a8c3491a3c42d393d355a451013dd3b687e5ac82b237ade6f939d779"} err="failed to get container status \"33c23d26a8c3491a3c42d393d355a451013dd3b687e5ac82b237ade6f939d779\": rpc error: code = NotFound desc = could not find container \"33c23d26a8c3491a3c42d393d355a451013dd3b687e5ac82b237ade6f939d779\": container with ID starting with 33c23d26a8c3491a3c42d393d355a451013dd3b687e5ac82b237ade6f939d779 not found: ID does not exist"
Jul 28 19:41:34.630157 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.630122    2561 scope.go:117] "RemoveContainer" containerID="9b1f232dd8c7de230cf3faee177d3cc82ed430675f75dc10ef91f56e87c1c1c9"
Jul 28 19:41:34.637853 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.637837    2561 scope.go:117] "RemoveContainer" containerID="9b1f232dd8c7de230cf3faee177d3cc82ed430675f75dc10ef91f56e87c1c1c9"
Jul 28 19:41:34.638117 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:41:34.638098    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b1f232dd8c7de230cf3faee177d3cc82ed430675f75dc10ef91f56e87c1c1c9\": container with ID starting with 9b1f232dd8c7de230cf3faee177d3cc82ed430675f75dc10ef91f56e87c1c1c9 not found: ID does not exist" containerID="9b1f232dd8c7de230cf3faee177d3cc82ed430675f75dc10ef91f56e87c1c1c9"
Jul 28 19:41:34.638173 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.638122    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b1f232dd8c7de230cf3faee177d3cc82ed430675f75dc10ef91f56e87c1c1c9"} err="failed to get container status \"9b1f232dd8c7de230cf3faee177d3cc82ed430675f75dc10ef91f56e87c1c1c9\": rpc error: code = NotFound desc = could not find container \"9b1f232dd8c7de230cf3faee177d3cc82ed430675f75dc10ef91f56e87c1c1c9\": container with ID starting with 9b1f232dd8c7de230cf3faee177d3cc82ed430675f75dc10ef91f56e87c1c1c9 not found: ID does not exist"
Jul 28 19:41:34.638173 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.638138    2561 scope.go:117] "RemoveContainer" containerID="dc771a7905edad2c3a0ef622e2bec24a6b294e61f7cf959252a1be8fc4d9c87e"
Jul 28 19:41:34.640210 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.640190    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce69b884-f0d2-4571-9438-ecba796aa12c-openshift-service-ca\") pod \"ce69b884-f0d2-4571-9438-ecba796aa12c\" (UID: \"ce69b884-f0d2-4571-9438-ecba796aa12c\") "
Jul 28 19:41:34.640315 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.640241    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfsm7\" (UniqueName: \"kubernetes.io/projected/ce69b884-f0d2-4571-9438-ecba796aa12c-kube-api-access-vfsm7\") pod \"ce69b884-f0d2-4571-9438-ecba796aa12c\" (UID: \"ce69b884-f0d2-4571-9438-ecba796aa12c\") "
Jul 28 19:41:34.640315 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.640292    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce69b884-f0d2-4571-9438-ecba796aa12c-maas-api-tls\") pod \"ce69b884-f0d2-4571-9438-ecba796aa12c\" (UID: \"ce69b884-f0d2-4571-9438-ecba796aa12c\") "
Jul 28 19:41:34.640593 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.640551    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40708c57-108b-4f97-9dc5-c67e45c09ce9-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:34.640593 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.640575    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0973213-7f08-4896-9d60-1252e654a06d-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:34.640593 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.640589    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9shgd\" (UniqueName: \"kubernetes.io/projected/40708c57-108b-4f97-9dc5-c67e45c09ce9-kube-api-access-9shgd\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:34.640824 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.640602    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzv25\" (UniqueName: \"kubernetes.io/projected/d0973213-7f08-4896-9d60-1252e654a06d-kube-api-access-mzv25\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:34.640824 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.640599    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce69b884-f0d2-4571-9438-ecba796aa12c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ce69b884-f0d2-4571-9438-ecba796aa12c" (UID: "ce69b884-f0d2-4571-9438-ecba796aa12c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:41:34.642803 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.642772    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce69b884-f0d2-4571-9438-ecba796aa12c-kube-api-access-vfsm7" (OuterVolumeSpecName: "kube-api-access-vfsm7") pod "ce69b884-f0d2-4571-9438-ecba796aa12c" (UID: "ce69b884-f0d2-4571-9438-ecba796aa12c"). InnerVolumeSpecName "kube-api-access-vfsm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:41:34.643018 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.642992    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce69b884-f0d2-4571-9438-ecba796aa12c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ce69b884-f0d2-4571-9438-ecba796aa12c" (UID: "ce69b884-f0d2-4571-9438-ecba796aa12c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:41:34.643981 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.643962    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"]
Jul 28 19:41:34.647326 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.647251    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e55e7d-7f97c6f96d-6xldd"]
Jul 28 19:41:34.647597 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.647582    2561 scope.go:117] "RemoveContainer" containerID="dc771a7905edad2c3a0ef622e2bec24a6b294e61f7cf959252a1be8fc4d9c87e"
Jul 28 19:41:34.647855 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:41:34.647837    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc771a7905edad2c3a0ef622e2bec24a6b294e61f7cf959252a1be8fc4d9c87e\": container with ID starting with dc771a7905edad2c3a0ef622e2bec24a6b294e61f7cf959252a1be8fc4d9c87e not found: ID does not exist" containerID="dc771a7905edad2c3a0ef622e2bec24a6b294e61f7cf959252a1be8fc4d9c87e"
Jul 28 19:41:34.647901 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.647862    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc771a7905edad2c3a0ef622e2bec24a6b294e61f7cf959252a1be8fc4d9c87e"} err="failed to get container status \"dc771a7905edad2c3a0ef622e2bec24a6b294e61f7cf959252a1be8fc4d9c87e\": rpc error: code = NotFound desc = could not find container \"dc771a7905edad2c3a0ef622e2bec24a6b294e61f7cf959252a1be8fc4d9c87e\": container with ID starting with dc771a7905edad2c3a0ef622e2bec24a6b294e61f7cf959252a1be8fc4d9c87e not found: ID does not exist"
Jul 28 19:41:34.656953 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.656933    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"]
Jul 28 19:41:34.660436 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.660417    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e55e7d-55bccfc5bd-g9494"]
Jul 28 19:41:34.741867 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.741843    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfsm7\" (UniqueName: \"kubernetes.io/projected/ce69b884-f0d2-4571-9438-ecba796aa12c-kube-api-access-vfsm7\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:34.741867 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.741865    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce69b884-f0d2-4571-9438-ecba796aa12c-maas-api-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:34.742029 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.741874    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce69b884-f0d2-4571-9438-ecba796aa12c-openshift-service-ca\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:34.942601 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.942571    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"]
Jul 28 19:41:34.946775 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:34.946752    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e55e7d-7b9f9986b6-cdm2k"]
Jul 28 19:41:36.609928 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:36.609889    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40708c57-108b-4f97-9dc5-c67e45c09ce9" path="/var/lib/kubelet/pods/40708c57-108b-4f97-9dc5-c67e45c09ce9/volumes"
Jul 28 19:41:36.610538 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:36.610505    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce69b884-f0d2-4571-9438-ecba796aa12c" path="/var/lib/kubelet/pods/ce69b884-f0d2-4571-9438-ecba796aa12c/volumes"
Jul 28 19:41:36.610916 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:36.610899    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0973213-7f08-4896-9d60-1252e654a06d" path="/var/lib/kubelet/pods/d0973213-7f08-4896-9d60-1252e654a06d/volumes"
Jul 28 19:41:37.344995 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.344948    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4" podUID="99460cfb-2fae-4c23-b93f-2b782944b44c" containerName="main" containerID="cri-o://1bda8158f3f58d53906393f1cca8c0a48c6d35a122799c5c567fddd980222460" gracePeriod=45
Jul 28 19:41:37.604564 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.604514    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:41:37.640679 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.640654    2561 generic.go:356] "Generic (PLEG): container finished" podID="99460cfb-2fae-4c23-b93f-2b782944b44c" containerID="1bda8158f3f58d53906393f1cca8c0a48c6d35a122799c5c567fddd980222460" exitCode=0
Jul 28 19:41:37.641045 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.640711    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"
Jul 28 19:41:37.641045 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.640731    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4" event={"ID":"99460cfb-2fae-4c23-b93f-2b782944b44c","Type":"ContainerDied","Data":"1bda8158f3f58d53906393f1cca8c0a48c6d35a122799c5c567fddd980222460"}
Jul 28 19:41:37.641045 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.640765    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4" event={"ID":"99460cfb-2fae-4c23-b93f-2b782944b44c","Type":"ContainerDied","Data":"e693fb488c7572fd456294ab19b811c53bf91a1f28adac359ef8e4140df13e03"}
Jul 28 19:41:37.641045 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.640788    2561 scope.go:117] "RemoveContainer" containerID="1bda8158f3f58d53906393f1cca8c0a48c6d35a122799c5c567fddd980222460"
Jul 28 19:41:37.649273 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.649255    2561 scope.go:117] "RemoveContainer" containerID="1bda8158f3f58d53906393f1cca8c0a48c6d35a122799c5c567fddd980222460"
Jul 28 19:41:37.649578 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:41:37.649556    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bda8158f3f58d53906393f1cca8c0a48c6d35a122799c5c567fddd980222460\": container with ID starting with 1bda8158f3f58d53906393f1cca8c0a48c6d35a122799c5c567fddd980222460 not found: ID does not exist" containerID="1bda8158f3f58d53906393f1cca8c0a48c6d35a122799c5c567fddd980222460"
Jul 28 19:41:37.649640 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.649586    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bda8158f3f58d53906393f1cca8c0a48c6d35a122799c5c567fddd980222460"} err="failed to get container status \"1bda8158f3f58d53906393f1cca8c0a48c6d35a122799c5c567fddd980222460\": rpc error: code = NotFound desc = could not find container \"1bda8158f3f58d53906393f1cca8c0a48c6d35a122799c5c567fddd980222460\": container with ID starting with 1bda8158f3f58d53906393f1cca8c0a48c6d35a122799c5c567fddd980222460 not found: ID does not exist"
Jul 28 19:41:37.664979 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.664957    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-dshm\") pod \"99460cfb-2fae-4c23-b93f-2b782944b44c\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") "
Jul 28 19:41:37.665065 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.664988    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-model-cache\") pod \"99460cfb-2fae-4c23-b93f-2b782944b44c\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") "
Jul 28 19:41:37.665065 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.665031    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-home\") pod \"99460cfb-2fae-4c23-b93f-2b782944b44c\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") "
Jul 28 19:41:37.665065 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.665064    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-tmp-dir\") pod \"99460cfb-2fae-4c23-b93f-2b782944b44c\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") "
Jul 28 19:41:37.665233 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.665084    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvq7k\" (UniqueName: \"kubernetes.io/projected/99460cfb-2fae-4c23-b93f-2b782944b44c-kube-api-access-pvq7k\") pod \"99460cfb-2fae-4c23-b93f-2b782944b44c\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") "
Jul 28 19:41:37.665233 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.665140    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99460cfb-2fae-4c23-b93f-2b782944b44c-tls-certs\") pod \"99460cfb-2fae-4c23-b93f-2b782944b44c\" (UID: \"99460cfb-2fae-4c23-b93f-2b782944b44c\") "
Jul 28 19:41:37.665330 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.665254    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-model-cache" (OuterVolumeSpecName: "model-cache") pod "99460cfb-2fae-4c23-b93f-2b782944b44c" (UID: "99460cfb-2fae-4c23-b93f-2b782944b44c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:37.665388 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.665332    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-home" (OuterVolumeSpecName: "home") pod "99460cfb-2fae-4c23-b93f-2b782944b44c" (UID: "99460cfb-2fae-4c23-b93f-2b782944b44c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:37.665388 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.665347    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "99460cfb-2fae-4c23-b93f-2b782944b44c" (UID: "99460cfb-2fae-4c23-b93f-2b782944b44c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:37.665562 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.665545    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-model-cache\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:37.665620 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.665566    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-home\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:37.665620 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.665579    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-tmp-dir\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:37.667225 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.667203    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99460cfb-2fae-4c23-b93f-2b782944b44c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "99460cfb-2fae-4c23-b93f-2b782944b44c" (UID: "99460cfb-2fae-4c23-b93f-2b782944b44c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:41:37.667398 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.667377    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-dshm" (OuterVolumeSpecName: "dshm") pod "99460cfb-2fae-4c23-b93f-2b782944b44c" (UID: "99460cfb-2fae-4c23-b93f-2b782944b44c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:41:37.667533 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.667416    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99460cfb-2fae-4c23-b93f-2b782944b44c-kube-api-access-pvq7k" (OuterVolumeSpecName: "kube-api-access-pvq7k") pod "99460cfb-2fae-4c23-b93f-2b782944b44c" (UID: "99460cfb-2fae-4c23-b93f-2b782944b44c"). InnerVolumeSpecName "kube-api-access-pvq7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:41:37.766068 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.766043    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99460cfb-2fae-4c23-b93f-2b782944b44c-tls-certs\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:37.766068 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.766066    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99460cfb-2fae-4c23-b93f-2b782944b44c-dshm\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:37.766190 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.766075    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvq7k\" (UniqueName: \"kubernetes.io/projected/99460cfb-2fae-4c23-b93f-2b782944b44c-kube-api-access-pvq7k\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:37.962767 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.962738    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"]
Jul 28 19:41:37.966168 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:37.966145    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e55e7d/test-model-e55e7d-kserve-5544d5b4f9-n74z4"]
Jul 28 19:41:38.603932 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:38.603903    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99460cfb-2fae-4c23-b93f-2b782944b44c" path="/var/lib/kubelet/pods/99460cfb-2fae-4c23-b93f-2b782944b44c/volumes"
Jul 28 19:41:45.734964 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:45.734885    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"]
Jul 28 19:41:45.735519 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:45.735138    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b" podUID="23d55773-35ce-43c7-a817-b47f98b8adc9" containerName="maas-api" containerID="cri-o://540d0b16535fbb05553ad9cfffa21e162c5bce943af7a34b9ab845927cfdb5f5" gracePeriod=30
Jul 28 19:41:45.790996 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:45.790968    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"]
Jul 28 19:41:45.791233 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:45.791180    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf" podUID="e571dbc9-da84-4715-a7df-8c18a00cda90" containerName="payload-processing" containerID="cri-o://57d425cb92edd9ace1a3e2d65691b8eb0e35b737feb7b2afafe6b93f00d32e73" gracePeriod=30
Jul 28 19:41:45.797215 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:45.797192    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"]
Jul 28 19:41:45.797392 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:45.797369    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx" podUID="24b484e6-5c27-4bab-9bed-029ded6b2e4e" containerName="payload-pre-processing" containerID="cri-o://b223cf0526c7718c1a42e05274954ba2ef40824aa2e4871e84b053863295bce7" gracePeriod=30
Jul 28 19:41:46.007644 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.007414    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"
Jul 28 19:41:46.033706 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.033681    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23d55773-35ce-43c7-a817-b47f98b8adc9-openshift-service-ca\") pod \"23d55773-35ce-43c7-a817-b47f98b8adc9\" (UID: \"23d55773-35ce-43c7-a817-b47f98b8adc9\") "
Jul 28 19:41:46.033844 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.033780    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4mr8\" (UniqueName: \"kubernetes.io/projected/23d55773-35ce-43c7-a817-b47f98b8adc9-kube-api-access-v4mr8\") pod \"23d55773-35ce-43c7-a817-b47f98b8adc9\" (UID: \"23d55773-35ce-43c7-a817-b47f98b8adc9\") "
Jul 28 19:41:46.033844 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.033821    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23d55773-35ce-43c7-a817-b47f98b8adc9-maas-api-tls\") pod \"23d55773-35ce-43c7-a817-b47f98b8adc9\" (UID: \"23d55773-35ce-43c7-a817-b47f98b8adc9\") "
Jul 28 19:41:46.034323 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.034105    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23d55773-35ce-43c7-a817-b47f98b8adc9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "23d55773-35ce-43c7-a817-b47f98b8adc9" (UID: "23d55773-35ce-43c7-a817-b47f98b8adc9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:41:46.036133 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.036109    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23d55773-35ce-43c7-a817-b47f98b8adc9-kube-api-access-v4mr8" (OuterVolumeSpecName: "kube-api-access-v4mr8") pod "23d55773-35ce-43c7-a817-b47f98b8adc9" (UID: "23d55773-35ce-43c7-a817-b47f98b8adc9"). InnerVolumeSpecName "kube-api-access-v4mr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:41:46.036433 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.036411    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23d55773-35ce-43c7-a817-b47f98b8adc9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "23d55773-35ce-43c7-a817-b47f98b8adc9" (UID: "23d55773-35ce-43c7-a817-b47f98b8adc9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:41:46.043906 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.043882    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"
Jul 28 19:41:46.078382 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.078361    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"
Jul 28 19:41:46.134725 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.134705    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn5sg\" (UniqueName: \"kubernetes.io/projected/e571dbc9-da84-4715-a7df-8c18a00cda90-kube-api-access-jn5sg\") pod \"e571dbc9-da84-4715-a7df-8c18a00cda90\" (UID: \"e571dbc9-da84-4715-a7df-8c18a00cda90\") "
Jul 28 19:41:46.134829 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.134754    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e571dbc9-da84-4715-a7df-8c18a00cda90-plugins-config-volume\") pod \"e571dbc9-da84-4715-a7df-8c18a00cda90\" (UID: \"e571dbc9-da84-4715-a7df-8c18a00cda90\") "
Jul 28 19:41:46.134829 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.134813    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24b484e6-5c27-4bab-9bed-029ded6b2e4e-plugins-config-volume\") pod \"24b484e6-5c27-4bab-9bed-029ded6b2e4e\" (UID: \"24b484e6-5c27-4bab-9bed-029ded6b2e4e\") "
Jul 28 19:41:46.134943 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.134839    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9csn6\" (UniqueName: \"kubernetes.io/projected/24b484e6-5c27-4bab-9bed-029ded6b2e4e-kube-api-access-9csn6\") pod \"24b484e6-5c27-4bab-9bed-029ded6b2e4e\" (UID: \"24b484e6-5c27-4bab-9bed-029ded6b2e4e\") "
Jul 28 19:41:46.135000 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.134985    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23d55773-35ce-43c7-a817-b47f98b8adc9-openshift-service-ca\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:46.135066 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.135000    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4mr8\" (UniqueName: \"kubernetes.io/projected/23d55773-35ce-43c7-a817-b47f98b8adc9-kube-api-access-v4mr8\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:46.135066 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.135014    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23d55773-35ce-43c7-a817-b47f98b8adc9-maas-api-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:46.135177 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.135153    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24b484e6-5c27-4bab-9bed-029ded6b2e4e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "24b484e6-5c27-4bab-9bed-029ded6b2e4e" (UID: "24b484e6-5c27-4bab-9bed-029ded6b2e4e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:41:46.135251 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.135228    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e571dbc9-da84-4715-a7df-8c18a00cda90-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e571dbc9-da84-4715-a7df-8c18a00cda90" (UID: "e571dbc9-da84-4715-a7df-8c18a00cda90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:41:46.136678 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.136657    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e571dbc9-da84-4715-a7df-8c18a00cda90-kube-api-access-jn5sg" (OuterVolumeSpecName: "kube-api-access-jn5sg") pod "e571dbc9-da84-4715-a7df-8c18a00cda90" (UID: "e571dbc9-da84-4715-a7df-8c18a00cda90"). InnerVolumeSpecName "kube-api-access-jn5sg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:41:46.136759 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.136739    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24b484e6-5c27-4bab-9bed-029ded6b2e4e-kube-api-access-9csn6" (OuterVolumeSpecName: "kube-api-access-9csn6") pod "24b484e6-5c27-4bab-9bed-029ded6b2e4e" (UID: "24b484e6-5c27-4bab-9bed-029ded6b2e4e"). InnerVolumeSpecName "kube-api-access-9csn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:41:46.236183 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.236129    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24b484e6-5c27-4bab-9bed-029ded6b2e4e-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:46.236183 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.236149    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9csn6\" (UniqueName: \"kubernetes.io/projected/24b484e6-5c27-4bab-9bed-029ded6b2e4e-kube-api-access-9csn6\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:46.236183 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.236159    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn5sg\" (UniqueName: \"kubernetes.io/projected/e571dbc9-da84-4715-a7df-8c18a00cda90-kube-api-access-jn5sg\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:46.236183 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.236168    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e571dbc9-da84-4715-a7df-8c18a00cda90-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:41:46.684686 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.684656    2561 generic.go:356] "Generic (PLEG): container finished" podID="24b484e6-5c27-4bab-9bed-029ded6b2e4e" containerID="b223cf0526c7718c1a42e05274954ba2ef40824aa2e4871e84b053863295bce7" exitCode=0
Jul 28 19:41:46.684790 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.684699    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"
Jul 28 19:41:46.684790 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.684703    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx" event={"ID":"24b484e6-5c27-4bab-9bed-029ded6b2e4e","Type":"ContainerDied","Data":"b223cf0526c7718c1a42e05274954ba2ef40824aa2e4871e84b053863295bce7"}
Jul 28 19:41:46.684790 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.684749    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx" event={"ID":"24b484e6-5c27-4bab-9bed-029ded6b2e4e","Type":"ContainerDied","Data":"95aaf157885000d4b23af4c42f124615d637dfde2908fcf54784354290b819c0"}
Jul 28 19:41:46.684790 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.684771    2561 scope.go:117] "RemoveContainer" containerID="b223cf0526c7718c1a42e05274954ba2ef40824aa2e4871e84b053863295bce7"
Jul 28 19:41:46.686079 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.686061    2561 generic.go:356] "Generic (PLEG): container finished" podID="23d55773-35ce-43c7-a817-b47f98b8adc9" containerID="540d0b16535fbb05553ad9cfffa21e162c5bce943af7a34b9ab845927cfdb5f5" exitCode=0
Jul 28 19:41:46.686176 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.686087    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b" event={"ID":"23d55773-35ce-43c7-a817-b47f98b8adc9","Type":"ContainerDied","Data":"540d0b16535fbb05553ad9cfffa21e162c5bce943af7a34b9ab845927cfdb5f5"}
Jul 28 19:41:46.686176 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.686117    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"
Jul 28 19:41:46.686176 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.686135    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b" event={"ID":"23d55773-35ce-43c7-a817-b47f98b8adc9","Type":"ContainerDied","Data":"3c58623c826914ed12487edc4169d9749fc8586c9417d4d54c3cfeaa0caf03e0"}
Jul 28 19:41:46.687757 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.687731    2561 generic.go:356] "Generic (PLEG): container finished" podID="e571dbc9-da84-4715-a7df-8c18a00cda90" containerID="57d425cb92edd9ace1a3e2d65691b8eb0e35b737feb7b2afafe6b93f00d32e73" exitCode=0
Jul 28 19:41:46.687860 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.687805    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"
Jul 28 19:41:46.687925 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.687805    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf" event={"ID":"e571dbc9-da84-4715-a7df-8c18a00cda90","Type":"ContainerDied","Data":"57d425cb92edd9ace1a3e2d65691b8eb0e35b737feb7b2afafe6b93f00d32e73"}
Jul 28 19:41:46.687986 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.687946    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf" event={"ID":"e571dbc9-da84-4715-a7df-8c18a00cda90","Type":"ContainerDied","Data":"aea18e7849b68a83f70c8088730a57bf47b5cd9e2fcc08c39d329e656726a873"}
Jul 28 19:41:46.693881 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.693866    2561 scope.go:117] "RemoveContainer" containerID="b223cf0526c7718c1a42e05274954ba2ef40824aa2e4871e84b053863295bce7"
Jul 28 19:41:46.694173 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:41:46.694154    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b223cf0526c7718c1a42e05274954ba2ef40824aa2e4871e84b053863295bce7\": container with ID starting with b223cf0526c7718c1a42e05274954ba2ef40824aa2e4871e84b053863295bce7 not found: ID does not exist" containerID="b223cf0526c7718c1a42e05274954ba2ef40824aa2e4871e84b053863295bce7"
Jul 28 19:41:46.694252 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.694183    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b223cf0526c7718c1a42e05274954ba2ef40824aa2e4871e84b053863295bce7"} err="failed to get container status \"b223cf0526c7718c1a42e05274954ba2ef40824aa2e4871e84b053863295bce7\": rpc error: code = NotFound desc = could not find container \"b223cf0526c7718c1a42e05274954ba2ef40824aa2e4871e84b053863295bce7\": container with ID starting with b223cf0526c7718c1a42e05274954ba2ef40824aa2e4871e84b053863295bce7 not found: ID does not exist"
Jul 28 19:41:46.694252 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.694205    2561 scope.go:117] "RemoveContainer" containerID="540d0b16535fbb05553ad9cfffa21e162c5bce943af7a34b9ab845927cfdb5f5"
Jul 28 19:41:46.701825 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.701807    2561 scope.go:117] "RemoveContainer" containerID="540d0b16535fbb05553ad9cfffa21e162c5bce943af7a34b9ab845927cfdb5f5"
Jul 28 19:41:46.702078 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:41:46.702050    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"540d0b16535fbb05553ad9cfffa21e162c5bce943af7a34b9ab845927cfdb5f5\": container with ID starting with 540d0b16535fbb05553ad9cfffa21e162c5bce943af7a34b9ab845927cfdb5f5 not found: ID does not exist" containerID="540d0b16535fbb05553ad9cfffa21e162c5bce943af7a34b9ab845927cfdb5f5"
Jul 28 19:41:46.702130 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.702076    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"540d0b16535fbb05553ad9cfffa21e162c5bce943af7a34b9ab845927cfdb5f5"} err="failed to get container status \"540d0b16535fbb05553ad9cfffa21e162c5bce943af7a34b9ab845927cfdb5f5\": rpc error: code = NotFound desc = could not find container \"540d0b16535fbb05553ad9cfffa21e162c5bce943af7a34b9ab845927cfdb5f5\": container with ID starting with 540d0b16535fbb05553ad9cfffa21e162c5bce943af7a34b9ab845927cfdb5f5 not found: ID does not exist"
Jul 28 19:41:46.702130 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.702092    2561 scope.go:117] "RemoveContainer" containerID="57d425cb92edd9ace1a3e2d65691b8eb0e35b737feb7b2afafe6b93f00d32e73"
Jul 28 19:41:46.706340 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.706319    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"]
Jul 28 19:41:46.710218 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.710200    2561 scope.go:117] "RemoveContainer" containerID="57d425cb92edd9ace1a3e2d65691b8eb0e35b737feb7b2afafe6b93f00d32e73"
Jul 28 19:41:46.710886 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:41:46.710862    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57d425cb92edd9ace1a3e2d65691b8eb0e35b737feb7b2afafe6b93f00d32e73\": container with ID starting with 57d425cb92edd9ace1a3e2d65691b8eb0e35b737feb7b2afafe6b93f00d32e73 not found: ID does not exist" containerID="57d425cb92edd9ace1a3e2d65691b8eb0e35b737feb7b2afafe6b93f00d32e73"
Jul 28 19:41:46.711000 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.710893    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57d425cb92edd9ace1a3e2d65691b8eb0e35b737feb7b2afafe6b93f00d32e73"} err="failed to get container status \"57d425cb92edd9ace1a3e2d65691b8eb0e35b737feb7b2afafe6b93f00d32e73\": rpc error: code = NotFound desc = could not find container \"57d425cb92edd9ace1a3e2d65691b8eb0e35b737feb7b2afafe6b93f00d32e73\": container with ID starting with 57d425cb92edd9ace1a3e2d65691b8eb0e35b737feb7b2afafe6b93f00d32e73 not found: ID does not exist"
Jul 28 19:41:46.715505 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.715458    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5f6911-749657b9b6-6xx5b"]
Jul 28 19:41:46.727505 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.727472    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"]
Jul 28 19:41:46.731237 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.731218    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5f6911-697c994c56-w2ppx"]
Jul 28 19:41:46.741631 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.741612    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"]
Jul 28 19:41:46.747831 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:46.747813    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5f6911-56494db55b-pjsvf"]
Jul 28 19:41:48.603571 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:48.603535    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23d55773-35ce-43c7-a817-b47f98b8adc9" path="/var/lib/kubelet/pods/23d55773-35ce-43c7-a817-b47f98b8adc9/volumes"
Jul 28 19:41:48.604094 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:48.604068    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24b484e6-5c27-4bab-9bed-029ded6b2e4e" path="/var/lib/kubelet/pods/24b484e6-5c27-4bab-9bed-029ded6b2e4e/volumes"
Jul 28 19:41:48.604512 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:48.604464    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e571dbc9-da84-4715-a7df-8c18a00cda90" path="/var/lib/kubelet/pods/e571dbc9-da84-4715-a7df-8c18a00cda90/volumes"
Jul 28 19:41:56.216584 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.216549    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"]
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.216891    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24b484e6-5c27-4bab-9bed-029ded6b2e4e" containerName="payload-pre-processing"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.216904    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="24b484e6-5c27-4bab-9bed-029ded6b2e4e" containerName="payload-pre-processing"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.216925    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40708c57-108b-4f97-9dc5-c67e45c09ce9" containerName="payload-pre-processing"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.216932    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="40708c57-108b-4f97-9dc5-c67e45c09ce9" containerName="payload-pre-processing"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.216941    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e571dbc9-da84-4715-a7df-8c18a00cda90" containerName="payload-processing"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.216946    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e571dbc9-da84-4715-a7df-8c18a00cda90" containerName="payload-processing"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.216960    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99460cfb-2fae-4c23-b93f-2b782944b44c" containerName="main"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.216965    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="99460cfb-2fae-4c23-b93f-2b782944b44c" containerName="main"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.216971    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0973213-7f08-4896-9d60-1252e654a06d" containerName="payload-processing"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.216976    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0973213-7f08-4896-9d60-1252e654a06d" containerName="payload-processing"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.216982    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce69b884-f0d2-4571-9438-ecba796aa12c" containerName="maas-api"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.216988    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce69b884-f0d2-4571-9438-ecba796aa12c" containerName="maas-api"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.217005    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23d55773-35ce-43c7-a817-b47f98b8adc9" containerName="maas-api"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.217010    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="23d55773-35ce-43c7-a817-b47f98b8adc9" containerName="maas-api"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.217066    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="24b484e6-5c27-4bab-9bed-029ded6b2e4e" containerName="payload-pre-processing"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.217075    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="40708c57-108b-4f97-9dc5-c67e45c09ce9" containerName="payload-pre-processing"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.217081    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="23d55773-35ce-43c7-a817-b47f98b8adc9" containerName="maas-api"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.217088    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e571dbc9-da84-4715-a7df-8c18a00cda90" containerName="payload-processing"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.217094    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce69b884-f0d2-4571-9438-ecba796aa12c" containerName="maas-api"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.217102    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="99460cfb-2fae-4c23-b93f-2b782944b44c" containerName="main"
Jul 28 19:41:56.217201 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.217107    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0973213-7f08-4896-9d60-1252e654a06d" containerName="payload-processing"
Jul 28 19:41:56.222042 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.222023    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"
Jul 28 19:41:56.225070 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.225051    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-29254e-serving-cert\""
Jul 28 19:41:56.225158 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.225056    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:41:56.225966 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.225950    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z259w\""
Jul 28 19:41:56.226021 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.225950    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 19:41:56.230283 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.230260    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"]
Jul 28 19:41:56.319174 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.319152    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-maas-api-tls\") pod \"maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2\" (UID: \"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"
Jul 28 19:41:56.319257 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.319179    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt2n8\" (UniqueName: \"kubernetes.io/projected/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-kube-api-access-mt2n8\") pod \"maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2\" (UID: \"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"
Jul 28 19:41:56.319257 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.319222    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-openshift-service-ca\") pod \"maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2\" (UID: \"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"
Jul 28 19:41:56.419948 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.419927    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-openshift-service-ca\") pod \"maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2\" (UID: \"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"
Jul 28 19:41:56.420051 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.419989    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-maas-api-tls\") pod \"maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2\" (UID: \"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"
Jul 28 19:41:56.420051 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.420014    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt2n8\" (UniqueName: \"kubernetes.io/projected/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-kube-api-access-mt2n8\") pod \"maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2\" (UID: \"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"
Jul 28 19:41:56.420144 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:41:56.420127    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-serving-cert: secret "maas-api-e2e-api-a-29254e-serving-cert" not found
Jul 28 19:41:56.420203 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:41:56.420192    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-maas-api-tls podName:8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8 nodeName:}" failed. No retries permitted until 2026-07-28 19:41:56.920173105 +0000 UTC m=+2364.866657966 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-maas-api-tls") pod "maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2" (UID: "8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8") : secret "maas-api-e2e-api-a-29254e-serving-cert" not found
Jul 28 19:41:56.420540 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.420523    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-openshift-service-ca\") pod \"maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2\" (UID: \"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"
Jul 28 19:41:56.430449 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.430426    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt2n8\" (UniqueName: \"kubernetes.io/projected/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-kube-api-access-mt2n8\") pod \"maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2\" (UID: \"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"
Jul 28 19:41:56.917681 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.917650    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"]
Jul 28 19:41:56.921249 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.921229    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"
Jul 28 19:41:56.923927 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.923905    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-29254e\""
Jul 28 19:41:56.925090 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.925072    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-maas-api-tls\") pod \"maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2\" (UID: \"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"
Jul 28 19:41:56.928042 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.928019    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-maas-api-tls\") pod \"maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2\" (UID: \"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"
Jul 28 19:41:56.929651 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.929628    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"]
Jul 28 19:41:56.939974 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.939952    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"]
Jul 28 19:41:56.943755 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.943737    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"
Jul 28 19:41:56.946531 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.946507    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-29254e-dockercfg-zn92g\""
Jul 28 19:41:56.952004 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:56.951986    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"]
Jul 28 19:41:57.026079 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.026056    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qzng\" (UniqueName: \"kubernetes.io/projected/2f46048c-d6f0-4495-a60a-df3781b0de02-kube-api-access-7qzng\") pod \"payload-processing-e2e-api-a-29254e-598c69bdb-flqf2\" (UID: \"2f46048c-d6f0-4495-a60a-df3781b0de02\") " pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"
Jul 28 19:41:57.026175 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.026102    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97b84b86-7f88-496f-8ee5-fc02fa814f74-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2\" (UID: \"97b84b86-7f88-496f-8ee5-fc02fa814f74\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"
Jul 28 19:41:57.026221 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.026191    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tq8m\" (UniqueName: \"kubernetes.io/projected/97b84b86-7f88-496f-8ee5-fc02fa814f74-kube-api-access-4tq8m\") pod \"payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2\" (UID: \"97b84b86-7f88-496f-8ee5-fc02fa814f74\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"
Jul 28 19:41:57.026258 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.026235    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f46048c-d6f0-4495-a60a-df3781b0de02-plugins-config-volume\") pod \"payload-processing-e2e-api-a-29254e-598c69bdb-flqf2\" (UID: \"2f46048c-d6f0-4495-a60a-df3781b0de02\") " pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"
Jul 28 19:41:57.127599 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.127570    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tq8m\" (UniqueName: \"kubernetes.io/projected/97b84b86-7f88-496f-8ee5-fc02fa814f74-kube-api-access-4tq8m\") pod \"payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2\" (UID: \"97b84b86-7f88-496f-8ee5-fc02fa814f74\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"
Jul 28 19:41:57.127763 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.127623    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f46048c-d6f0-4495-a60a-df3781b0de02-plugins-config-volume\") pod \"payload-processing-e2e-api-a-29254e-598c69bdb-flqf2\" (UID: \"2f46048c-d6f0-4495-a60a-df3781b0de02\") " pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"
Jul 28 19:41:57.127763 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.127648    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qzng\" (UniqueName: \"kubernetes.io/projected/2f46048c-d6f0-4495-a60a-df3781b0de02-kube-api-access-7qzng\") pod \"payload-processing-e2e-api-a-29254e-598c69bdb-flqf2\" (UID: \"2f46048c-d6f0-4495-a60a-df3781b0de02\") " pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"
Jul 28 19:41:57.127763 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.127687    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97b84b86-7f88-496f-8ee5-fc02fa814f74-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2\" (UID: \"97b84b86-7f88-496f-8ee5-fc02fa814f74\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"
Jul 28 19:41:57.128392 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.128369    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97b84b86-7f88-496f-8ee5-fc02fa814f74-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2\" (UID: \"97b84b86-7f88-496f-8ee5-fc02fa814f74\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"
Jul 28 19:41:57.128514 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.128370    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f46048c-d6f0-4495-a60a-df3781b0de02-plugins-config-volume\") pod \"payload-processing-e2e-api-a-29254e-598c69bdb-flqf2\" (UID: \"2f46048c-d6f0-4495-a60a-df3781b0de02\") " pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"
Jul 28 19:41:57.133786 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.133769    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"
Jul 28 19:41:57.135603 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.135577    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qzng\" (UniqueName: \"kubernetes.io/projected/2f46048c-d6f0-4495-a60a-df3781b0de02-kube-api-access-7qzng\") pod \"payload-processing-e2e-api-a-29254e-598c69bdb-flqf2\" (UID: \"2f46048c-d6f0-4495-a60a-df3781b0de02\") " pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"
Jul 28 19:41:57.135996 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.135974    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tq8m\" (UniqueName: \"kubernetes.io/projected/97b84b86-7f88-496f-8ee5-fc02fa814f74-kube-api-access-4tq8m\") pod \"payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2\" (UID: \"97b84b86-7f88-496f-8ee5-fc02fa814f74\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"
Jul 28 19:41:57.243031 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.242999    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"
Jul 28 19:41:57.255042 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.255018    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"
Jul 28 19:41:57.266118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.266094    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"]
Jul 28 19:41:57.268873 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:41:57.268802    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d92ff6d_db02_4f1a_b3b5_92c1cae8deb8.slice/crio-0552105cfe52ab790c4392668fb93f1a00947ce612230a661e3001262628d3ad WatchSource:0}: Error finding container 0552105cfe52ab790c4392668fb93f1a00947ce612230a661e3001262628d3ad: Status 404 returned error can't find the container with id 0552105cfe52ab790c4392668fb93f1a00947ce612230a661e3001262628d3ad
Jul 28 19:41:57.270390 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.270365    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:41:57.391007 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.390977    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"]
Jul 28 19:41:57.392912 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:41:57.392874    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97b84b86_7f88_496f_8ee5_fc02fa814f74.slice/crio-58b185b986146716689d396a88bea43404ff0dc69591bd38091f8e3c2dd01460 WatchSource:0}: Error finding container 58b185b986146716689d396a88bea43404ff0dc69591bd38091f8e3c2dd01460: Status 404 returned error can't find the container with id 58b185b986146716689d396a88bea43404ff0dc69591bd38091f8e3c2dd01460
Jul 28 19:41:57.418708 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.418689    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"]
Jul 28 19:41:57.423560 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:41:57.423537    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f46048c_d6f0_4495_a60a_df3781b0de02.slice/crio-c12339031d378b9b68c83f82a1d6fcd897bd92bfb88d4ba7c96a33adc0a3461e WatchSource:0}: Error finding container c12339031d378b9b68c83f82a1d6fcd897bd92bfb88d4ba7c96a33adc0a3461e: Status 404 returned error can't find the container with id c12339031d378b9b68c83f82a1d6fcd897bd92bfb88d4ba7c96a33adc0a3461e
Jul 28 19:41:57.745359 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.745331    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2" event={"ID":"2f46048c-d6f0-4495-a60a-df3781b0de02","Type":"ContainerStarted","Data":"1a1fe7bdc12f55f87b22c6470b377a813f87303acbe4c196f0c8e6276a7935d4"}
Jul 28 19:41:57.745504 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.745379    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2" event={"ID":"2f46048c-d6f0-4495-a60a-df3781b0de02","Type":"ContainerStarted","Data":"c12339031d378b9b68c83f82a1d6fcd897bd92bfb88d4ba7c96a33adc0a3461e"}
Jul 28 19:41:57.745504 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.745422    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"
Jul 28 19:41:57.747061 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.747034    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2" event={"ID":"97b84b86-7f88-496f-8ee5-fc02fa814f74","Type":"ContainerStarted","Data":"4124c9d95016005af535ba3f9a5911bc70cf2c88a6e9188684628477f11550b2"}
Jul 28 19:41:57.747175 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.747094    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"
Jul 28 19:41:57.747175 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.747122    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2" event={"ID":"97b84b86-7f88-496f-8ee5-fc02fa814f74","Type":"ContainerStarted","Data":"58b185b986146716689d396a88bea43404ff0dc69591bd38091f8e3c2dd01460"}
Jul 28 19:41:57.748762 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.748736    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2" event={"ID":"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8","Type":"ContainerStarted","Data":"0552105cfe52ab790c4392668fb93f1a00947ce612230a661e3001262628d3ad"}
Jul 28 19:41:57.772172 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.772131    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2" podStartSLOduration=1.772118203 podStartE2EDuration="1.772118203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:41:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:41:57.77068214 +0000 UTC m=+2365.717167008" watchObservedRunningTime="2026-07-28 19:41:57.772118203 +0000 UTC m=+2365.718603071"
Jul 28 19:41:57.796146 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:57.796108    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2" podStartSLOduration=1.7960966090000001 podStartE2EDuration="1.796096609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:41:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:41:57.793739201 +0000 UTC m=+2365.740224090" watchObservedRunningTime="2026-07-28 19:41:57.796096609 +0000 UTC m=+2365.742581477"
Jul 28 19:41:58.754856 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:58.754817    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2" event={"ID":"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8","Type":"ContainerStarted","Data":"53f251ffb03057b3b5fd5d9cfd2942f96bf7364f5c3d404da562fdccb4d8c4aa"}
Jul 28 19:41:58.755254 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:58.755126    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"
Jul 28 19:41:58.770689 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:41:58.770640    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2" podStartSLOduration=2.355541163 podStartE2EDuration="2.770626059s" totalImagesPullingTime="415.084896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:41:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:41:58.769555484 +0000 UTC m=+2366.716040365" watchObservedRunningTime="2026-07-28 19:41:58.770626059 +0000 UTC m=+2366.717110926"
Jul 28 19:42:05.768630 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:05.768596    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"
Jul 28 19:42:08.757304 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:08.757276    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"
Jul 28 19:42:08.757723 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:08.757469    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"
Jul 28 19:42:24.395500 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:24.395460    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"]
Jul 28 19:42:24.399138 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:24.399116    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"
Jul 28 19:42:24.401521 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:24.401479    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-7a6162-serving-cert\""
Jul 28 19:42:24.410303 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:24.410272    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"]
Jul 28 19:42:24.567798 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:24.567768    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91fd0025-1f8d-4486-9903-086b0983a2c5-openshift-service-ca\") pod \"maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9\" (UID: \"91fd0025-1f8d-4486-9903-086b0983a2c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"
Jul 28 19:42:24.567940 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:24.567829    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91fd0025-1f8d-4486-9903-086b0983a2c5-maas-api-tls\") pod \"maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9\" (UID: \"91fd0025-1f8d-4486-9903-086b0983a2c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"
Jul 28 19:42:24.567940 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:24.567859    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8lsb\" (UniqueName: \"kubernetes.io/projected/91fd0025-1f8d-4486-9903-086b0983a2c5-kube-api-access-x8lsb\") pod \"maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9\" (UID: \"91fd0025-1f8d-4486-9903-086b0983a2c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"
Jul 28 19:42:24.669170 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:24.669104    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91fd0025-1f8d-4486-9903-086b0983a2c5-maas-api-tls\") pod \"maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9\" (UID: \"91fd0025-1f8d-4486-9903-086b0983a2c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"
Jul 28 19:42:24.669170 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:24.669133    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8lsb\" (UniqueName: \"kubernetes.io/projected/91fd0025-1f8d-4486-9903-086b0983a2c5-kube-api-access-x8lsb\") pod \"maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9\" (UID: \"91fd0025-1f8d-4486-9903-086b0983a2c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"
Jul 28 19:42:24.669320 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:24.669193    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91fd0025-1f8d-4486-9903-086b0983a2c5-openshift-service-ca\") pod \"maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9\" (UID: \"91fd0025-1f8d-4486-9903-086b0983a2c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"
Jul 28 19:42:24.669320 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:42:24.669257    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-serving-cert: secret "maas-api-e2e-api-b-7a6162-serving-cert" not found
Jul 28 19:42:24.669320 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:42:24.669319    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91fd0025-1f8d-4486-9903-086b0983a2c5-maas-api-tls podName:91fd0025-1f8d-4486-9903-086b0983a2c5 nodeName:}" failed. No retries permitted until 2026-07-28 19:42:25.169301163 +0000 UTC m=+2393.115786009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/91fd0025-1f8d-4486-9903-086b0983a2c5-maas-api-tls") pod "maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9" (UID: "91fd0025-1f8d-4486-9903-086b0983a2c5") : secret "maas-api-e2e-api-b-7a6162-serving-cert" not found
Jul 28 19:42:24.669938 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:24.669919    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91fd0025-1f8d-4486-9903-086b0983a2c5-openshift-service-ca\") pod \"maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9\" (UID: \"91fd0025-1f8d-4486-9903-086b0983a2c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"
Jul 28 19:42:24.681261 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:24.681237    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8lsb\" (UniqueName: \"kubernetes.io/projected/91fd0025-1f8d-4486-9903-086b0983a2c5-kube-api-access-x8lsb\") pod \"maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9\" (UID: \"91fd0025-1f8d-4486-9903-086b0983a2c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"
Jul 28 19:42:25.157373 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.157334    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"]
Jul 28 19:42:25.161687 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.161668    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"
Jul 28 19:42:25.164161 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.164135    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-7a6162-dockercfg-pcbjt\""
Jul 28 19:42:25.164161 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.164134    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-7a6162\""
Jul 28 19:42:25.170814 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.169316    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"]
Jul 28 19:42:25.175253 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.175215    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91fd0025-1f8d-4486-9903-086b0983a2c5-maas-api-tls\") pod \"maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9\" (UID: \"91fd0025-1f8d-4486-9903-086b0983a2c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"
Jul 28 19:42:25.179627 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.179604    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91fd0025-1f8d-4486-9903-086b0983a2c5-maas-api-tls\") pod \"maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9\" (UID: \"91fd0025-1f8d-4486-9903-086b0983a2c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"
Jul 28 19:42:25.188832 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.188793    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"]
Jul 28 19:42:25.192344 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.192327    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"
Jul 28 19:42:25.202271 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.202246    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"]
Jul 28 19:42:25.276043 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.276005    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bmb2\" (UniqueName: \"kubernetes.io/projected/b92aa8a6-03ba-430e-8b8d-ebaa0e01e968-kube-api-access-9bmb2\") pod \"payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg\" (UID: \"b92aa8a6-03ba-430e-8b8d-ebaa0e01e968\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"
Jul 28 19:42:25.276043 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.276049    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b92aa8a6-03ba-430e-8b8d-ebaa0e01e968-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg\" (UID: \"b92aa8a6-03ba-430e-8b8d-ebaa0e01e968\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"
Jul 28 19:42:25.321037 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.321006    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"
Jul 28 19:42:25.377655 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.377628    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bmb2\" (UniqueName: \"kubernetes.io/projected/b92aa8a6-03ba-430e-8b8d-ebaa0e01e968-kube-api-access-9bmb2\") pod \"payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg\" (UID: \"b92aa8a6-03ba-430e-8b8d-ebaa0e01e968\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"
Jul 28 19:42:25.378448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.378404    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b92aa8a6-03ba-430e-8b8d-ebaa0e01e968-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg\" (UID: \"b92aa8a6-03ba-430e-8b8d-ebaa0e01e968\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"
Jul 28 19:42:25.380021 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.380001    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlsfj\" (UniqueName: \"kubernetes.io/projected/64f44119-dd1f-4805-b693-b1338483bcbe-kube-api-access-dlsfj\") pod \"payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4\" (UID: \"64f44119-dd1f-4805-b693-b1338483bcbe\") " pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"
Jul 28 19:42:25.380740 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.380304    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64f44119-dd1f-4805-b693-b1338483bcbe-plugins-config-volume\") pod \"payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4\" (UID: \"64f44119-dd1f-4805-b693-b1338483bcbe\") " pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"
Jul 28 19:42:25.380964 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.379911    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b92aa8a6-03ba-430e-8b8d-ebaa0e01e968-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg\" (UID: \"b92aa8a6-03ba-430e-8b8d-ebaa0e01e968\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"
Jul 28 19:42:25.386454 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.386431    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bmb2\" (UniqueName: \"kubernetes.io/projected/b92aa8a6-03ba-430e-8b8d-ebaa0e01e968-kube-api-access-9bmb2\") pod \"payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg\" (UID: \"b92aa8a6-03ba-430e-8b8d-ebaa0e01e968\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"
Jul 28 19:42:25.476855 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.476813    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"
Jul 28 19:42:25.482329 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.482306    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlsfj\" (UniqueName: \"kubernetes.io/projected/64f44119-dd1f-4805-b693-b1338483bcbe-kube-api-access-dlsfj\") pod \"payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4\" (UID: \"64f44119-dd1f-4805-b693-b1338483bcbe\") " pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"
Jul 28 19:42:25.482441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.482379    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64f44119-dd1f-4805-b693-b1338483bcbe-plugins-config-volume\") pod \"payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4\" (UID: \"64f44119-dd1f-4805-b693-b1338483bcbe\") " pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"
Jul 28 19:42:25.483094 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.483075    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64f44119-dd1f-4805-b693-b1338483bcbe-plugins-config-volume\") pod \"payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4\" (UID: \"64f44119-dd1f-4805-b693-b1338483bcbe\") " pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"
Jul 28 19:42:25.495506 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.495461    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlsfj\" (UniqueName: \"kubernetes.io/projected/64f44119-dd1f-4805-b693-b1338483bcbe-kube-api-access-dlsfj\") pod \"payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4\" (UID: \"64f44119-dd1f-4805-b693-b1338483bcbe\") " pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"
Jul 28 19:42:25.504174 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.504152    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"
Jul 28 19:42:25.619937 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.619912    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"]
Jul 28 19:42:25.622325 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:42:25.622293    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb92aa8a6_03ba_430e_8b8d_ebaa0e01e968.slice/crio-b224404420ee0ede81d42f3133d525a55caeb79c17b37760aa99ebfbeeb87089 WatchSource:0}: Error finding container b224404420ee0ede81d42f3133d525a55caeb79c17b37760aa99ebfbeeb87089: Status 404 returned error can't find the container with id b224404420ee0ede81d42f3133d525a55caeb79c17b37760aa99ebfbeeb87089
Jul 28 19:42:25.663903 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.663875    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"]
Jul 28 19:42:25.665652 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:42:25.665472    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64f44119_dd1f_4805_b693_b1338483bcbe.slice/crio-a744a43b8a9deadd4d67e0669226c5b973584e8bb40838ec5d08a156a7303804 WatchSource:0}: Error finding container a744a43b8a9deadd4d67e0669226c5b973584e8bb40838ec5d08a156a7303804: Status 404 returned error can't find the container with id a744a43b8a9deadd4d67e0669226c5b973584e8bb40838ec5d08a156a7303804
Jul 28 19:42:25.668886 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.668418    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"]
Jul 28 19:42:25.669806 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:42:25.669783    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91fd0025_1f8d_4486_9903_086b0983a2c5.slice/crio-d8512c03cfcc491853c0b98f2dbb4380b1893487dbeec00a96ea12eb9686d1f3 WatchSource:0}: Error finding container d8512c03cfcc491853c0b98f2dbb4380b1893487dbeec00a96ea12eb9686d1f3: Status 404 returned error can't find the container with id d8512c03cfcc491853c0b98f2dbb4380b1893487dbeec00a96ea12eb9686d1f3
Jul 28 19:42:25.894811 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.894773    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9" event={"ID":"91fd0025-1f8d-4486-9903-086b0983a2c5","Type":"ContainerStarted","Data":"d8512c03cfcc491853c0b98f2dbb4380b1893487dbeec00a96ea12eb9686d1f3"}
Jul 28 19:42:25.896551 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.896519    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg" event={"ID":"b92aa8a6-03ba-430e-8b8d-ebaa0e01e968","Type":"ContainerStarted","Data":"2bec4d9da46a8e560212f07376301bcd54f04c4bc7f244345e278fb3f641c9af"}
Jul 28 19:42:25.896698 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.896574    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg" event={"ID":"b92aa8a6-03ba-430e-8b8d-ebaa0e01e968","Type":"ContainerStarted","Data":"b224404420ee0ede81d42f3133d525a55caeb79c17b37760aa99ebfbeeb87089"}
Jul 28 19:42:25.896698 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.896652    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"
Jul 28 19:42:25.898330 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.898291    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4" event={"ID":"64f44119-dd1f-4805-b693-b1338483bcbe","Type":"ContainerStarted","Data":"0d5e7bb62d88271a354978f043e50438a54ab336bc6739380ad4aa56037018e6"}
Jul 28 19:42:25.898459 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.898356    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"
Jul 28 19:42:25.898459 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.898381    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4" event={"ID":"64f44119-dd1f-4805-b693-b1338483bcbe","Type":"ContainerStarted","Data":"a744a43b8a9deadd4d67e0669226c5b973584e8bb40838ec5d08a156a7303804"}
Jul 28 19:42:25.913185 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.913101    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg" podStartSLOduration=0.913084633 podStartE2EDuration="913.084633ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:25.911168598 +0000 UTC m=+2393.857653470" watchObservedRunningTime="2026-07-28 19:42:25.913084633 +0000 UTC m=+2393.859569481"
Jul 28 19:42:25.929047 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:25.929005    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4" podStartSLOduration=0.928995119 podStartE2EDuration="928.995119ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:25.926253699 +0000 UTC m=+2393.872738591" watchObservedRunningTime="2026-07-28 19:42:25.928995119 +0000 UTC m=+2393.875479987"
Jul 28 19:42:26.908686 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:26.908610    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9" event={"ID":"91fd0025-1f8d-4486-9903-086b0983a2c5","Type":"ContainerStarted","Data":"a4f9431013a6d5a9bbba6f697185fe12d431025c794c5bfeca47696e8b4564d5"}
Jul 28 19:42:26.909176 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:26.908797    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"
Jul 28 19:42:26.925116 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:26.925070    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9" podStartSLOduration=2.630933774 podStartE2EDuration="2.925054574s" totalImagesPullingTime="294.1208ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:26.922836759 +0000 UTC m=+2394.869321652" watchObservedRunningTime="2026-07-28 19:42:26.925054574 +0000 UTC m=+2394.871539442"
Jul 28 19:42:32.615556 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:32.615517    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:42:32.622570 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:32.622540    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:42:32.627681 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:32.627658    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:42:32.634853 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:32.634832    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:42:32.919597 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:32.919541    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"
Jul 28 19:42:36.911154 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:36.911122    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"
Jul 28 19:42:36.911551 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:42:36.911190    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"
Jul 28 19:43:07.890961 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:07.890853    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"]
Jul 28 19:43:07.891618 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:07.891210    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2" podUID="8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8" containerName="maas-api" containerID="cri-o://53f251ffb03057b3b5fd5d9cfd2942f96bf7364f5c3d404da562fdccb4d8c4aa" gracePeriod=30
Jul 28 19:43:07.946341 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:07.946313    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"]
Jul 28 19:43:07.946556 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:07.946532    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2" podUID="2f46048c-d6f0-4495-a60a-df3781b0de02" containerName="payload-processing" containerID="cri-o://1a1fe7bdc12f55f87b22c6470b377a813f87303acbe4c196f0c8e6276a7935d4" gracePeriod=30
Jul 28 19:43:07.954829 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:07.954794    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"]
Jul 28 19:43:07.955225 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:07.955175    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2" podUID="97b84b86-7f88-496f-8ee5-fc02fa814f74" containerName="payload-pre-processing" containerID="cri-o://4124c9d95016005af535ba3f9a5911bc70cf2c88a6e9188684628477f11550b2" gracePeriod=30
Jul 28 19:43:08.118616 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.118515    2561 generic.go:356] "Generic (PLEG): container finished" podID="2f46048c-d6f0-4495-a60a-df3781b0de02" containerID="1a1fe7bdc12f55f87b22c6470b377a813f87303acbe4c196f0c8e6276a7935d4" exitCode=0
Jul 28 19:43:08.118717 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.118629    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2" event={"ID":"2f46048c-d6f0-4495-a60a-df3781b0de02","Type":"ContainerDied","Data":"1a1fe7bdc12f55f87b22c6470b377a813f87303acbe4c196f0c8e6276a7935d4"}
Jul 28 19:43:08.122044 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.120599    2561 generic.go:356] "Generic (PLEG): container finished" podID="97b84b86-7f88-496f-8ee5-fc02fa814f74" containerID="4124c9d95016005af535ba3f9a5911bc70cf2c88a6e9188684628477f11550b2" exitCode=0
Jul 28 19:43:08.122044 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.120756    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2" event={"ID":"97b84b86-7f88-496f-8ee5-fc02fa814f74","Type":"ContainerDied","Data":"4124c9d95016005af535ba3f9a5911bc70cf2c88a6e9188684628477f11550b2"}
Jul 28 19:43:08.122882 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.122854    2561 generic.go:356] "Generic (PLEG): container finished" podID="8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8" containerID="53f251ffb03057b3b5fd5d9cfd2942f96bf7364f5c3d404da562fdccb4d8c4aa" exitCode=0
Jul 28 19:43:08.122975 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.122912    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2" event={"ID":"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8","Type":"ContainerDied","Data":"53f251ffb03057b3b5fd5d9cfd2942f96bf7364f5c3d404da562fdccb4d8c4aa"}
Jul 28 19:43:08.230332 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.230306    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"
Jul 28 19:43:08.233472 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.233455    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"
Jul 28 19:43:08.245240 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.245223    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"
Jul 28 19:43:08.335127 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.335099    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97b84b86-7f88-496f-8ee5-fc02fa814f74-plugins-config-volume\") pod \"97b84b86-7f88-496f-8ee5-fc02fa814f74\" (UID: \"97b84b86-7f88-496f-8ee5-fc02fa814f74\") "
Jul 28 19:43:08.335272 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.335150    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f46048c-d6f0-4495-a60a-df3781b0de02-plugins-config-volume\") pod \"2f46048c-d6f0-4495-a60a-df3781b0de02\" (UID: \"2f46048c-d6f0-4495-a60a-df3781b0de02\") "
Jul 28 19:43:08.335272 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.335247    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-openshift-service-ca\") pod \"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8\" (UID: \"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8\") "
Jul 28 19:43:08.335432 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.335412    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qzng\" (UniqueName: \"kubernetes.io/projected/2f46048c-d6f0-4495-a60a-df3781b0de02-kube-api-access-7qzng\") pod \"2f46048c-d6f0-4495-a60a-df3781b0de02\" (UID: \"2f46048c-d6f0-4495-a60a-df3781b0de02\") "
Jul 28 19:43:08.335530 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.335449    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tq8m\" (UniqueName: \"kubernetes.io/projected/97b84b86-7f88-496f-8ee5-fc02fa814f74-kube-api-access-4tq8m\") pod \"97b84b86-7f88-496f-8ee5-fc02fa814f74\" (UID: \"97b84b86-7f88-496f-8ee5-fc02fa814f74\") "
Jul 28 19:43:08.335530 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.335503    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt2n8\" (UniqueName: \"kubernetes.io/projected/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-kube-api-access-mt2n8\") pod \"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8\" (UID: \"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8\") "
Jul 28 19:43:08.335637 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.335537    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-maas-api-tls\") pod \"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8\" (UID: \"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8\") "
Jul 28 19:43:08.335637 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.335565    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97b84b86-7f88-496f-8ee5-fc02fa814f74-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "97b84b86-7f88-496f-8ee5-fc02fa814f74" (UID: "97b84b86-7f88-496f-8ee5-fc02fa814f74"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:43:08.335637 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.335618    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8" (UID: "8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:43:08.335779 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.335633    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f46048c-d6f0-4495-a60a-df3781b0de02-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2f46048c-d6f0-4495-a60a-df3781b0de02" (UID: "2f46048c-d6f0-4495-a60a-df3781b0de02"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:43:08.335896 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.335877    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-openshift-service-ca\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:08.335957 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.335901    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97b84b86-7f88-496f-8ee5-fc02fa814f74-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:08.335957 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.335916    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f46048c-d6f0-4495-a60a-df3781b0de02-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:08.337442 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.337421    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f46048c-d6f0-4495-a60a-df3781b0de02-kube-api-access-7qzng" (OuterVolumeSpecName: "kube-api-access-7qzng") pod "2f46048c-d6f0-4495-a60a-df3781b0de02" (UID: "2f46048c-d6f0-4495-a60a-df3781b0de02"). InnerVolumeSpecName "kube-api-access-7qzng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:43:08.337553 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.337465    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-kube-api-access-mt2n8" (OuterVolumeSpecName: "kube-api-access-mt2n8") pod "8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8" (UID: "8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8"). InnerVolumeSpecName "kube-api-access-mt2n8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:43:08.337904 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.337880    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8" (UID: "8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:43:08.337967 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.337887    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97b84b86-7f88-496f-8ee5-fc02fa814f74-kube-api-access-4tq8m" (OuterVolumeSpecName: "kube-api-access-4tq8m") pod "97b84b86-7f88-496f-8ee5-fc02fa814f74" (UID: "97b84b86-7f88-496f-8ee5-fc02fa814f74"). InnerVolumeSpecName "kube-api-access-4tq8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:43:08.436350 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.436289    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qzng\" (UniqueName: \"kubernetes.io/projected/2f46048c-d6f0-4495-a60a-df3781b0de02-kube-api-access-7qzng\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:08.436350 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.436315    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tq8m\" (UniqueName: \"kubernetes.io/projected/97b84b86-7f88-496f-8ee5-fc02fa814f74-kube-api-access-4tq8m\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:08.436350 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.436331    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt2n8\" (UniqueName: \"kubernetes.io/projected/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-kube-api-access-mt2n8\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:08.436350 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:08.436343    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8-maas-api-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:09.129139 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:09.129108    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"
Jul 28 19:43:09.129555 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:09.129113    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2" event={"ID":"8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8","Type":"ContainerDied","Data":"0552105cfe52ab790c4392668fb93f1a00947ce612230a661e3001262628d3ad"}
Jul 28 19:43:09.129555 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:09.129285    2561 scope.go:117] "RemoveContainer" containerID="53f251ffb03057b3b5fd5d9cfd2942f96bf7364f5c3d404da562fdccb4d8c4aa"
Jul 28 19:43:09.130823 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:09.130704    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"
Jul 28 19:43:09.130823 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:09.130714    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2" event={"ID":"2f46048c-d6f0-4495-a60a-df3781b0de02","Type":"ContainerDied","Data":"c12339031d378b9b68c83f82a1d6fcd897bd92bfb88d4ba7c96a33adc0a3461e"}
Jul 28 19:43:09.132428 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:09.132399    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2" event={"ID":"97b84b86-7f88-496f-8ee5-fc02fa814f74","Type":"ContainerDied","Data":"58b185b986146716689d396a88bea43404ff0dc69591bd38091f8e3c2dd01460"}
Jul 28 19:43:09.132572 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:09.132428    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"
Jul 28 19:43:09.139275 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:09.139259    2561 scope.go:117] "RemoveContainer" containerID="1a1fe7bdc12f55f87b22c6470b377a813f87303acbe4c196f0c8e6276a7935d4"
Jul 28 19:43:09.147162 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:09.147143    2561 scope.go:117] "RemoveContainer" containerID="4124c9d95016005af535ba3f9a5911bc70cf2c88a6e9188684628477f11550b2"
Jul 28 19:43:09.150640 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:09.150613    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"]
Jul 28 19:43:09.154455 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:09.154434    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-29254e-598c69bdb-flqf2"]
Jul 28 19:43:09.164616 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:09.164570    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"]
Jul 28 19:43:09.168557 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:09.168536    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-29254e-6457fddd4c-bhrc2"]
Jul 28 19:43:09.178405 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:09.178382    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"]
Jul 28 19:43:09.181820 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:09.181802    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-29254e-695f866964-l6tc2"]
Jul 28 19:43:10.605466 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:10.605431    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f46048c-d6f0-4495-a60a-df3781b0de02" path="/var/lib/kubelet/pods/2f46048c-d6f0-4495-a60a-df3781b0de02/volumes"
Jul 28 19:43:10.606049 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:10.606023    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8" path="/var/lib/kubelet/pods/8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8/volumes"
Jul 28 19:43:10.606572 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:10.606554    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97b84b86-7f88-496f-8ee5-fc02fa814f74" path="/var/lib/kubelet/pods/97b84b86-7f88-496f-8ee5-fc02fa814f74/volumes"
Jul 28 19:43:30.383408 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.383363    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"]
Jul 28 19:43:30.383866 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.383670    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9" podUID="91fd0025-1f8d-4486-9903-086b0983a2c5" containerName="maas-api" containerID="cri-o://a4f9431013a6d5a9bbba6f697185fe12d431025c794c5bfeca47696e8b4564d5" gracePeriod=30
Jul 28 19:43:30.439529 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.439470    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"]
Jul 28 19:43:30.440161 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.440100    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4" podUID="64f44119-dd1f-4805-b693-b1338483bcbe" containerName="payload-processing" containerID="cri-o://0d5e7bb62d88271a354978f043e50438a54ab336bc6739380ad4aa56037018e6" gracePeriod=30
Jul 28 19:43:30.444848 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.444824    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"]
Jul 28 19:43:30.445057 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.445023    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg" podUID="b92aa8a6-03ba-430e-8b8d-ebaa0e01e968" containerName="payload-pre-processing" containerID="cri-o://2bec4d9da46a8e560212f07376301bcd54f04c4bc7f244345e278fb3f641c9af" gracePeriod=30
Jul 28 19:43:30.691637 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.691610    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"
Jul 28 19:43:30.725494 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.725459    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"
Jul 28 19:43:30.731018 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.731001    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"
Jul 28 19:43:30.813492 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.813459    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlsfj\" (UniqueName: \"kubernetes.io/projected/64f44119-dd1f-4805-b693-b1338483bcbe-kube-api-access-dlsfj\") pod \"64f44119-dd1f-4805-b693-b1338483bcbe\" (UID: \"64f44119-dd1f-4805-b693-b1338483bcbe\") "
Jul 28 19:43:30.813627 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.813516    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b92aa8a6-03ba-430e-8b8d-ebaa0e01e968-plugins-config-volume\") pod \"b92aa8a6-03ba-430e-8b8d-ebaa0e01e968\" (UID: \"b92aa8a6-03ba-430e-8b8d-ebaa0e01e968\") "
Jul 28 19:43:30.813627 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.813600    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8lsb\" (UniqueName: \"kubernetes.io/projected/91fd0025-1f8d-4486-9903-086b0983a2c5-kube-api-access-x8lsb\") pod \"91fd0025-1f8d-4486-9903-086b0983a2c5\" (UID: \"91fd0025-1f8d-4486-9903-086b0983a2c5\") "
Jul 28 19:43:30.813627 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.813628    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64f44119-dd1f-4805-b693-b1338483bcbe-plugins-config-volume\") pod \"64f44119-dd1f-4805-b693-b1338483bcbe\" (UID: \"64f44119-dd1f-4805-b693-b1338483bcbe\") "
Jul 28 19:43:30.813780 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.813648    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91fd0025-1f8d-4486-9903-086b0983a2c5-openshift-service-ca\") pod \"91fd0025-1f8d-4486-9903-086b0983a2c5\" (UID: \"91fd0025-1f8d-4486-9903-086b0983a2c5\") "
Jul 28 19:43:30.813780 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.813665    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bmb2\" (UniqueName: \"kubernetes.io/projected/b92aa8a6-03ba-430e-8b8d-ebaa0e01e968-kube-api-access-9bmb2\") pod \"b92aa8a6-03ba-430e-8b8d-ebaa0e01e968\" (UID: \"b92aa8a6-03ba-430e-8b8d-ebaa0e01e968\") "
Jul 28 19:43:30.813780 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.813681    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91fd0025-1f8d-4486-9903-086b0983a2c5-maas-api-tls\") pod \"91fd0025-1f8d-4486-9903-086b0983a2c5\" (UID: \"91fd0025-1f8d-4486-9903-086b0983a2c5\") "
Jul 28 19:43:30.813992 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.813944    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b92aa8a6-03ba-430e-8b8d-ebaa0e01e968-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b92aa8a6-03ba-430e-8b8d-ebaa0e01e968" (UID: "b92aa8a6-03ba-430e-8b8d-ebaa0e01e968"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:43:30.814052 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.814025    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64f44119-dd1f-4805-b693-b1338483bcbe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "64f44119-dd1f-4805-b693-b1338483bcbe" (UID: "64f44119-dd1f-4805-b693-b1338483bcbe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:43:30.814107 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.814071    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91fd0025-1f8d-4486-9903-086b0983a2c5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "91fd0025-1f8d-4486-9903-086b0983a2c5" (UID: "91fd0025-1f8d-4486-9903-086b0983a2c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:43:30.815842 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.815822    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64f44119-dd1f-4805-b693-b1338483bcbe-kube-api-access-dlsfj" (OuterVolumeSpecName: "kube-api-access-dlsfj") pod "64f44119-dd1f-4805-b693-b1338483bcbe" (UID: "64f44119-dd1f-4805-b693-b1338483bcbe"). InnerVolumeSpecName "kube-api-access-dlsfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:43:30.816035 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.816018    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91fd0025-1f8d-4486-9903-086b0983a2c5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "91fd0025-1f8d-4486-9903-086b0983a2c5" (UID: "91fd0025-1f8d-4486-9903-086b0983a2c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:43:30.816217 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.816200    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91fd0025-1f8d-4486-9903-086b0983a2c5-kube-api-access-x8lsb" (OuterVolumeSpecName: "kube-api-access-x8lsb") pod "91fd0025-1f8d-4486-9903-086b0983a2c5" (UID: "91fd0025-1f8d-4486-9903-086b0983a2c5"). InnerVolumeSpecName "kube-api-access-x8lsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:43:30.816276 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.816218    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b92aa8a6-03ba-430e-8b8d-ebaa0e01e968-kube-api-access-9bmb2" (OuterVolumeSpecName: "kube-api-access-9bmb2") pod "b92aa8a6-03ba-430e-8b8d-ebaa0e01e968" (UID: "b92aa8a6-03ba-430e-8b8d-ebaa0e01e968"). InnerVolumeSpecName "kube-api-access-9bmb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:43:30.914841 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.914822    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bmb2\" (UniqueName: \"kubernetes.io/projected/b92aa8a6-03ba-430e-8b8d-ebaa0e01e968-kube-api-access-9bmb2\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:30.914841 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.914843    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91fd0025-1f8d-4486-9903-086b0983a2c5-maas-api-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:30.914983 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.914852    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlsfj\" (UniqueName: \"kubernetes.io/projected/64f44119-dd1f-4805-b693-b1338483bcbe-kube-api-access-dlsfj\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:30.914983 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.914861    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b92aa8a6-03ba-430e-8b8d-ebaa0e01e968-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:30.914983 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.914870    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8lsb\" (UniqueName: \"kubernetes.io/projected/91fd0025-1f8d-4486-9903-086b0983a2c5-kube-api-access-x8lsb\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:30.914983 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.914879    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64f44119-dd1f-4805-b693-b1338483bcbe-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:30.914983 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:30.914886    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91fd0025-1f8d-4486-9903-086b0983a2c5-openshift-service-ca\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:43:31.241575 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.241547    2561 generic.go:356] "Generic (PLEG): container finished" podID="91fd0025-1f8d-4486-9903-086b0983a2c5" containerID="a4f9431013a6d5a9bbba6f697185fe12d431025c794c5bfeca47696e8b4564d5" exitCode=0
Jul 28 19:43:31.241684 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.241583    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"
Jul 28 19:43:31.241684 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.241638    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9" event={"ID":"91fd0025-1f8d-4486-9903-086b0983a2c5","Type":"ContainerDied","Data":"a4f9431013a6d5a9bbba6f697185fe12d431025c794c5bfeca47696e8b4564d5"}
Jul 28 19:43:31.241684 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.241681    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9" event={"ID":"91fd0025-1f8d-4486-9903-086b0983a2c5","Type":"ContainerDied","Data":"d8512c03cfcc491853c0b98f2dbb4380b1893487dbeec00a96ea12eb9686d1f3"}
Jul 28 19:43:31.241823 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.241707    2561 scope.go:117] "RemoveContainer" containerID="a4f9431013a6d5a9bbba6f697185fe12d431025c794c5bfeca47696e8b4564d5"
Jul 28 19:43:31.242875 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.242850    2561 generic.go:356] "Generic (PLEG): container finished" podID="b92aa8a6-03ba-430e-8b8d-ebaa0e01e968" containerID="2bec4d9da46a8e560212f07376301bcd54f04c4bc7f244345e278fb3f641c9af" exitCode=0
Jul 28 19:43:31.242968 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.242896    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"
Jul 28 19:43:31.242968 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.242932    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg" event={"ID":"b92aa8a6-03ba-430e-8b8d-ebaa0e01e968","Type":"ContainerDied","Data":"2bec4d9da46a8e560212f07376301bcd54f04c4bc7f244345e278fb3f641c9af"}
Jul 28 19:43:31.243100 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.242970    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg" event={"ID":"b92aa8a6-03ba-430e-8b8d-ebaa0e01e968","Type":"ContainerDied","Data":"b224404420ee0ede81d42f3133d525a55caeb79c17b37760aa99ebfbeeb87089"}
Jul 28 19:43:31.244428 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.244401    2561 generic.go:356] "Generic (PLEG): container finished" podID="64f44119-dd1f-4805-b693-b1338483bcbe" containerID="0d5e7bb62d88271a354978f043e50438a54ab336bc6739380ad4aa56037018e6" exitCode=0
Jul 28 19:43:31.244566 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.244446    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"
Jul 28 19:43:31.244566 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.244472    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4" event={"ID":"64f44119-dd1f-4805-b693-b1338483bcbe","Type":"ContainerDied","Data":"0d5e7bb62d88271a354978f043e50438a54ab336bc6739380ad4aa56037018e6"}
Jul 28 19:43:31.244566 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.244560    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4" event={"ID":"64f44119-dd1f-4805-b693-b1338483bcbe","Type":"ContainerDied","Data":"a744a43b8a9deadd4d67e0669226c5b973584e8bb40838ec5d08a156a7303804"}
Jul 28 19:43:31.251870 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.251855    2561 scope.go:117] "RemoveContainer" containerID="a4f9431013a6d5a9bbba6f697185fe12d431025c794c5bfeca47696e8b4564d5"
Jul 28 19:43:31.252196 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:43:31.252173    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4f9431013a6d5a9bbba6f697185fe12d431025c794c5bfeca47696e8b4564d5\": container with ID starting with a4f9431013a6d5a9bbba6f697185fe12d431025c794c5bfeca47696e8b4564d5 not found: ID does not exist" containerID="a4f9431013a6d5a9bbba6f697185fe12d431025c794c5bfeca47696e8b4564d5"
Jul 28 19:43:31.252284 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.252210    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f9431013a6d5a9bbba6f697185fe12d431025c794c5bfeca47696e8b4564d5"} err="failed to get container status \"a4f9431013a6d5a9bbba6f697185fe12d431025c794c5bfeca47696e8b4564d5\": rpc error: code = NotFound desc = could not find container \"a4f9431013a6d5a9bbba6f697185fe12d431025c794c5bfeca47696e8b4564d5\": container with ID starting with a4f9431013a6d5a9bbba6f697185fe12d431025c794c5bfeca47696e8b4564d5 not found: ID does not exist"
Jul 28 19:43:31.252284 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.252235    2561 scope.go:117] "RemoveContainer" containerID="2bec4d9da46a8e560212f07376301bcd54f04c4bc7f244345e278fb3f641c9af"
Jul 28 19:43:31.260026 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.260010    2561 scope.go:117] "RemoveContainer" containerID="2bec4d9da46a8e560212f07376301bcd54f04c4bc7f244345e278fb3f641c9af"
Jul 28 19:43:31.260261 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:43:31.260242    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bec4d9da46a8e560212f07376301bcd54f04c4bc7f244345e278fb3f641c9af\": container with ID starting with 2bec4d9da46a8e560212f07376301bcd54f04c4bc7f244345e278fb3f641c9af not found: ID does not exist" containerID="2bec4d9da46a8e560212f07376301bcd54f04c4bc7f244345e278fb3f641c9af"
Jul 28 19:43:31.260311 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.260268    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bec4d9da46a8e560212f07376301bcd54f04c4bc7f244345e278fb3f641c9af"} err="failed to get container status \"2bec4d9da46a8e560212f07376301bcd54f04c4bc7f244345e278fb3f641c9af\": rpc error: code = NotFound desc = could not find container \"2bec4d9da46a8e560212f07376301bcd54f04c4bc7f244345e278fb3f641c9af\": container with ID starting with 2bec4d9da46a8e560212f07376301bcd54f04c4bc7f244345e278fb3f641c9af not found: ID does not exist"
Jul 28 19:43:31.260311 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.260284    2561 scope.go:117] "RemoveContainer" containerID="0d5e7bb62d88271a354978f043e50438a54ab336bc6739380ad4aa56037018e6"
Jul 28 19:43:31.267003 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.266984    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"]
Jul 28 19:43:31.267744 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.267730    2561 scope.go:117] "RemoveContainer" containerID="0d5e7bb62d88271a354978f043e50438a54ab336bc6739380ad4aa56037018e6"
Jul 28 19:43:31.267982 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:43:31.267966    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d5e7bb62d88271a354978f043e50438a54ab336bc6739380ad4aa56037018e6\": container with ID starting with 0d5e7bb62d88271a354978f043e50438a54ab336bc6739380ad4aa56037018e6 not found: ID does not exist" containerID="0d5e7bb62d88271a354978f043e50438a54ab336bc6739380ad4aa56037018e6"
Jul 28 19:43:31.268027 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.267988    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d5e7bb62d88271a354978f043e50438a54ab336bc6739380ad4aa56037018e6"} err="failed to get container status \"0d5e7bb62d88271a354978f043e50438a54ab336bc6739380ad4aa56037018e6\": rpc error: code = NotFound desc = could not find container \"0d5e7bb62d88271a354978f043e50438a54ab336bc6739380ad4aa56037018e6\": container with ID starting with 0d5e7bb62d88271a354978f043e50438a54ab336bc6739380ad4aa56037018e6 not found: ID does not exist"
Jul 28 19:43:31.271845 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.271824    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-7a6162-7f4b4b6dc6-jlffg"]
Jul 28 19:43:31.281044 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.281025    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"]
Jul 28 19:43:31.286404 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.286385    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7a6162-f5d9d8dbb-hsxg9"]
Jul 28 19:43:31.295831 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.295809    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"]
Jul 28 19:43:31.297105 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:31.297086    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7a6162-665c75996d-w4pn4"]
Jul 28 19:43:32.604786 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:32.604751    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64f44119-dd1f-4805-b693-b1338483bcbe" path="/var/lib/kubelet/pods/64f44119-dd1f-4805-b693-b1338483bcbe/volumes"
Jul 28 19:43:32.605235 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:32.605123    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91fd0025-1f8d-4486-9903-086b0983a2c5" path="/var/lib/kubelet/pods/91fd0025-1f8d-4486-9903-086b0983a2c5/volumes"
Jul 28 19:43:32.605430 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:32.605417    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b92aa8a6-03ba-430e-8b8d-ebaa0e01e968" path="/var/lib/kubelet/pods/b92aa8a6-03ba-430e-8b8d-ebaa0e01e968/volumes"
Jul 28 19:43:40.742134 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742099    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"]
Jul 28 19:43:40.742611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742442    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64f44119-dd1f-4805-b693-b1338483bcbe" containerName="payload-processing"
Jul 28 19:43:40.742611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742454    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="64f44119-dd1f-4805-b693-b1338483bcbe" containerName="payload-processing"
Jul 28 19:43:40.742611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742463    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8" containerName="maas-api"
Jul 28 19:43:40.742611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742469    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8" containerName="maas-api"
Jul 28 19:43:40.742611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742476    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f46048c-d6f0-4495-a60a-df3781b0de02" containerName="payload-processing"
Jul 28 19:43:40.742611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742504    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f46048c-d6f0-4495-a60a-df3781b0de02" containerName="payload-processing"
Jul 28 19:43:40.742611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742512    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b92aa8a6-03ba-430e-8b8d-ebaa0e01e968" containerName="payload-pre-processing"
Jul 28 19:43:40.742611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742518    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b92aa8a6-03ba-430e-8b8d-ebaa0e01e968" containerName="payload-pre-processing"
Jul 28 19:43:40.742611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742527    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97b84b86-7f88-496f-8ee5-fc02fa814f74" containerName="payload-pre-processing"
Jul 28 19:43:40.742611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742531    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="97b84b86-7f88-496f-8ee5-fc02fa814f74" containerName="payload-pre-processing"
Jul 28 19:43:40.742611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742543    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91fd0025-1f8d-4486-9903-086b0983a2c5" containerName="maas-api"
Jul 28 19:43:40.742611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742549    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="91fd0025-1f8d-4486-9903-086b0983a2c5" containerName="maas-api"
Jul 28 19:43:40.742611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742596    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="64f44119-dd1f-4805-b693-b1338483bcbe" containerName="payload-processing"
Jul 28 19:43:40.742611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742605    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d92ff6d-db02-4f1a-b3b5-92c1cae8deb8" containerName="maas-api"
Jul 28 19:43:40.742611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742610    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="91fd0025-1f8d-4486-9903-086b0983a2c5" containerName="maas-api"
Jul 28 19:43:40.742611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742616    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b92aa8a6-03ba-430e-8b8d-ebaa0e01e968" containerName="payload-pre-processing"
Jul 28 19:43:40.743159 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742622    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="97b84b86-7f88-496f-8ee5-fc02fa814f74" containerName="payload-pre-processing"
Jul 28 19:43:40.743159 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.742628    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f46048c-d6f0-4495-a60a-df3781b0de02" containerName="payload-processing"
Jul 28 19:43:40.745290 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.745273    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"
Jul 28 19:43:40.747946 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.747920    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-ec81e7-serving-cert\""
Jul 28 19:43:40.747946 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.747936    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 19:43:40.748137 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.747953    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 19:43:40.748137 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.748005    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z259w\""
Jul 28 19:43:40.761260 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.761238    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"]
Jul 28 19:43:40.791276 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.791256    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/381624bd-7373-48b0-9fb0-9b6b8d792790-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h\" (UID: \"381624bd-7373-48b0-9fb0-9b6b8d792790\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"
Jul 28 19:43:40.791391 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.791292    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/381624bd-7373-48b0-9fb0-9b6b8d792790-maas-api-tls\") pod \"maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h\" (UID: \"381624bd-7373-48b0-9fb0-9b6b8d792790\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"
Jul 28 19:43:40.791458 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.791390    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7f65\" (UniqueName: \"kubernetes.io/projected/381624bd-7373-48b0-9fb0-9b6b8d792790-kube-api-access-m7f65\") pod \"maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h\" (UID: \"381624bd-7373-48b0-9fb0-9b6b8d792790\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"
Jul 28 19:43:40.892602 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.892582    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/381624bd-7373-48b0-9fb0-9b6b8d792790-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h\" (UID: \"381624bd-7373-48b0-9fb0-9b6b8d792790\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"
Jul 28 19:43:40.892686 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.892611    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/381624bd-7373-48b0-9fb0-9b6b8d792790-maas-api-tls\") pod \"maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h\" (UID: \"381624bd-7373-48b0-9fb0-9b6b8d792790\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"
Jul 28 19:43:40.892686 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.892651    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7f65\" (UniqueName: \"kubernetes.io/projected/381624bd-7373-48b0-9fb0-9b6b8d792790-kube-api-access-m7f65\") pod \"maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h\" (UID: \"381624bd-7373-48b0-9fb0-9b6b8d792790\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"
Jul 28 19:43:40.892756 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:43:40.892736    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-serving-cert: secret "maas-api-e2e-shared-a-ec81e7-serving-cert" not found
Jul 28 19:43:40.892824 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:43:40.892806    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/381624bd-7373-48b0-9fb0-9b6b8d792790-maas-api-tls podName:381624bd-7373-48b0-9fb0-9b6b8d792790 nodeName:}" failed. No retries permitted until 2026-07-28 19:43:41.392788279 +0000 UTC m=+2469.339273136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/381624bd-7373-48b0-9fb0-9b6b8d792790-maas-api-tls") pod "maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h" (UID: "381624bd-7373-48b0-9fb0-9b6b8d792790") : secret "maas-api-e2e-shared-a-ec81e7-serving-cert" not found
Jul 28 19:43:40.893164 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.893145    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/381624bd-7373-48b0-9fb0-9b6b8d792790-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h\" (UID: \"381624bd-7373-48b0-9fb0-9b6b8d792790\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"
Jul 28 19:43:40.903791 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:40.903767    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7f65\" (UniqueName: \"kubernetes.io/projected/381624bd-7373-48b0-9fb0-9b6b8d792790-kube-api-access-m7f65\") pod \"maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h\" (UID: \"381624bd-7373-48b0-9fb0-9b6b8d792790\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"
Jul 28 19:43:41.398283 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.398259    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/381624bd-7373-48b0-9fb0-9b6b8d792790-maas-api-tls\") pod \"maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h\" (UID: \"381624bd-7373-48b0-9fb0-9b6b8d792790\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"
Jul 28 19:43:41.400469 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.400452    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/381624bd-7373-48b0-9fb0-9b6b8d792790-maas-api-tls\") pod \"maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h\" (UID: \"381624bd-7373-48b0-9fb0-9b6b8d792790\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"
Jul 28 19:43:41.511644 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.511621    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"]
Jul 28 19:43:41.515024 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.515010    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"
Jul 28 19:43:41.517884 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.517867    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-ec81e7\""
Jul 28 19:43:41.526104 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.526084    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"]
Jul 28 19:43:41.543624 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.543599    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"]
Jul 28 19:43:41.546715 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.546697    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"
Jul 28 19:43:41.551344 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.551328    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-ec81e7-dockercfg-qfkxn\""
Jul 28 19:43:41.562839 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.562817    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"]
Jul 28 19:43:41.599877 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.599857    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddb26822-8d56-4554-b097-efbc3e9a99bd-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9\" (UID: \"ddb26822-8d56-4554-b097-efbc3e9a99bd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"
Jul 28 19:43:41.599965 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.599912    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9597\" (UniqueName: \"kubernetes.io/projected/5911e907-05c1-45d8-aefc-984c393ff259-kube-api-access-j9597\") pod \"payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98\" (UID: \"5911e907-05c1-45d8-aefc-984c393ff259\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"
Jul 28 19:43:41.599965 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.599950    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5911e907-05c1-45d8-aefc-984c393ff259-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98\" (UID: \"5911e907-05c1-45d8-aefc-984c393ff259\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"
Jul 28 19:43:41.600044 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.599968    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpj7p\" (UniqueName: \"kubernetes.io/projected/ddb26822-8d56-4554-b097-efbc3e9a99bd-kube-api-access-hpj7p\") pod \"payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9\" (UID: \"ddb26822-8d56-4554-b097-efbc3e9a99bd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"
Jul 28 19:43:41.656284 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.656230    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"
Jul 28 19:43:41.700668 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.700640    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9597\" (UniqueName: \"kubernetes.io/projected/5911e907-05c1-45d8-aefc-984c393ff259-kube-api-access-j9597\") pod \"payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98\" (UID: \"5911e907-05c1-45d8-aefc-984c393ff259\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"
Jul 28 19:43:41.700793 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.700679    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5911e907-05c1-45d8-aefc-984c393ff259-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98\" (UID: \"5911e907-05c1-45d8-aefc-984c393ff259\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"
Jul 28 19:43:41.700793 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.700698    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpj7p\" (UniqueName: \"kubernetes.io/projected/ddb26822-8d56-4554-b097-efbc3e9a99bd-kube-api-access-hpj7p\") pod \"payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9\" (UID: \"ddb26822-8d56-4554-b097-efbc3e9a99bd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"
Jul 28 19:43:41.700793 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.700755    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddb26822-8d56-4554-b097-efbc3e9a99bd-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9\" (UID: \"ddb26822-8d56-4554-b097-efbc3e9a99bd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"
Jul 28 19:43:41.701405 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.701378    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5911e907-05c1-45d8-aefc-984c393ff259-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98\" (UID: \"5911e907-05c1-45d8-aefc-984c393ff259\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"
Jul 28 19:43:41.701537 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.701424    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddb26822-8d56-4554-b097-efbc3e9a99bd-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9\" (UID: \"ddb26822-8d56-4554-b097-efbc3e9a99bd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"
Jul 28 19:43:41.722108 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.721286    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9597\" (UniqueName: \"kubernetes.io/projected/5911e907-05c1-45d8-aefc-984c393ff259-kube-api-access-j9597\") pod \"payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98\" (UID: \"5911e907-05c1-45d8-aefc-984c393ff259\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"
Jul 28 19:43:41.724451 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.724403    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpj7p\" (UniqueName: \"kubernetes.io/projected/ddb26822-8d56-4554-b097-efbc3e9a99bd-kube-api-access-hpj7p\") pod \"payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9\" (UID: \"ddb26822-8d56-4554-b097-efbc3e9a99bd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"
Jul 28 19:43:41.815030 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.815001    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"]
Jul 28 19:43:41.817155 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:43:41.817127    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod381624bd_7373_48b0_9fb0_9b6b8d792790.slice/crio-354a2549ee5db229a3e684188de48696ee6413515d0b40c18b25907c4500a299 WatchSource:0}: Error finding container 354a2549ee5db229a3e684188de48696ee6413515d0b40c18b25907c4500a299: Status 404 returned error can't find the container with id 354a2549ee5db229a3e684188de48696ee6413515d0b40c18b25907c4500a299
Jul 28 19:43:41.826055 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.826038    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"
Jul 28 19:43:41.856173 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.856153    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"
Jul 28 19:43:41.967506 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.967367    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"]
Jul 28 19:43:41.971988 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:43:41.971952    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5911e907_05c1_45d8_aefc_984c393ff259.slice/crio-c3502eac87100e020ce79b52f918daa100fc7efe0f1ab160f6571f52991d8164 WatchSource:0}: Error finding container c3502eac87100e020ce79b52f918daa100fc7efe0f1ab160f6571f52991d8164: Status 404 returned error can't find the container with id c3502eac87100e020ce79b52f918daa100fc7efe0f1ab160f6571f52991d8164
Jul 28 19:43:41.998759 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:41.998734    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"]
Jul 28 19:43:42.009544 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:43:42.009515    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddb26822_8d56_4554_b097_efbc3e9a99bd.slice/crio-45366f8fc4c537f655924ca0bb184527d5c61cd03256bd287a2e44f22c9328a0 WatchSource:0}: Error finding container 45366f8fc4c537f655924ca0bb184527d5c61cd03256bd287a2e44f22c9328a0: Status 404 returned error can't find the container with id 45366f8fc4c537f655924ca0bb184527d5c61cd03256bd287a2e44f22c9328a0
Jul 28 19:43:42.295474 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:42.295449    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9" event={"ID":"ddb26822-8d56-4554-b097-efbc3e9a99bd","Type":"ContainerStarted","Data":"629df25e33cbb346c9f664e1ae5a45d9464b445d419c33e621931b5f4bc5250c"}
Jul 28 19:43:42.295594 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:42.295526    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9" event={"ID":"ddb26822-8d56-4554-b097-efbc3e9a99bd","Type":"ContainerStarted","Data":"45366f8fc4c537f655924ca0bb184527d5c61cd03256bd287a2e44f22c9328a0"}
Jul 28 19:43:42.295658 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:42.295597    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"
Jul 28 19:43:42.296972 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:42.296947    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98" event={"ID":"5911e907-05c1-45d8-aefc-984c393ff259","Type":"ContainerStarted","Data":"7b62c27578c5d87d51e52a34cf7cc6ec3ba74e20804c50d59f416a785d06ffbb"}
Jul 28 19:43:42.297091 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:42.296994    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"
Jul 28 19:43:42.297091 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:42.297017    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98" event={"ID":"5911e907-05c1-45d8-aefc-984c393ff259","Type":"ContainerStarted","Data":"c3502eac87100e020ce79b52f918daa100fc7efe0f1ab160f6571f52991d8164"}
Jul 28 19:43:42.298357 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:42.298339    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h" event={"ID":"381624bd-7373-48b0-9fb0-9b6b8d792790","Type":"ContainerStarted","Data":"985f192328a5f114a22b0b5a54d7bff3c07f5920ab25e1eeb1b7d13e0579be45"}
Jul 28 19:43:42.298468 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:42.298370    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h" event={"ID":"381624bd-7373-48b0-9fb0-9b6b8d792790","Type":"ContainerStarted","Data":"354a2549ee5db229a3e684188de48696ee6413515d0b40c18b25907c4500a299"}
Jul 28 19:43:42.298468 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:42.298418    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"
Jul 28 19:43:42.331304 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:42.331258    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9" podStartSLOduration=1.331238997 podStartE2EDuration="1.331238997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:43:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:43:42.32860443 +0000 UTC m=+2470.275089508" watchObservedRunningTime="2026-07-28 19:43:42.331238997 +0000 UTC m=+2470.277723866"
Jul 28 19:43:42.357539 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:42.357500    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98" podStartSLOduration=1.357475842 podStartE2EDuration="1.357475842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:43:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:43:42.355698018 +0000 UTC m=+2470.302182900" watchObservedRunningTime="2026-07-28 19:43:42.357475842 +0000 UTC m=+2470.303960688"
Jul 28 19:43:42.385106 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:42.385071    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h" podStartSLOduration=2.008401814 podStartE2EDuration="2.385058242s" totalImagesPullingTime="376.656428ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:43:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:43:42.384261585 +0000 UTC m=+2470.330746449" watchObservedRunningTime="2026-07-28 19:43:42.385058242 +0000 UTC m=+2470.331543109"
Jul 28 19:43:48.309331 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:48.309299    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"
Jul 28 19:43:53.305259 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:53.305223    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"
Jul 28 19:43:53.305687 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:43:53.305390    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"
Jul 28 19:44:08.866763 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:08.866729    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"]
Jul 28 19:44:08.870074 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:08.870052    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"
Jul 28 19:44:08.872604 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:08.872580    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-d66111-serving-cert\""
Jul 28 19:44:08.882866 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:08.882842    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"]
Jul 28 19:44:09.027323 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.027294    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-openshift-service-ca\") pod \"maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695\" (UID: \"bcac02cc-aa07-4c95-bd7b-6be5076b0c55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"
Jul 28 19:44:09.027513 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.027362    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-maas-api-tls\") pod \"maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695\" (UID: \"bcac02cc-aa07-4c95-bd7b-6be5076b0c55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"
Jul 28 19:44:09.027513 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.027448    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kzhv\" (UniqueName: \"kubernetes.io/projected/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-kube-api-access-7kzhv\") pod \"maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695\" (UID: \"bcac02cc-aa07-4c95-bd7b-6be5076b0c55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"
Jul 28 19:44:09.128010 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.127939    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kzhv\" (UniqueName: \"kubernetes.io/projected/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-kube-api-access-7kzhv\") pod \"maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695\" (UID: \"bcac02cc-aa07-4c95-bd7b-6be5076b0c55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"
Jul 28 19:44:09.128010 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.127976    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-openshift-service-ca\") pod \"maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695\" (UID: \"bcac02cc-aa07-4c95-bd7b-6be5076b0c55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"
Jul 28 19:44:09.128160 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.128039    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-maas-api-tls\") pod \"maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695\" (UID: \"bcac02cc-aa07-4c95-bd7b-6be5076b0c55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"
Jul 28 19:44:09.128160 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:44:09.128132    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-serving-cert: secret "maas-api-e2e-shared-b-d66111-serving-cert" not found
Jul 28 19:44:09.128222 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:44:09.128187    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-maas-api-tls podName:bcac02cc-aa07-4c95-bd7b-6be5076b0c55 nodeName:}" failed. No retries permitted until 2026-07-28 19:44:09.628171887 +0000 UTC m=+2497.574656732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-maas-api-tls") pod "maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695" (UID: "bcac02cc-aa07-4c95-bd7b-6be5076b0c55") : secret "maas-api-e2e-shared-b-d66111-serving-cert" not found
Jul 28 19:44:09.128700 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.128681    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-openshift-service-ca\") pod \"maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695\" (UID: \"bcac02cc-aa07-4c95-bd7b-6be5076b0c55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"
Jul 28 19:44:09.144965 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.144931    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kzhv\" (UniqueName: \"kubernetes.io/projected/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-kube-api-access-7kzhv\") pod \"maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695\" (UID: \"bcac02cc-aa07-4c95-bd7b-6be5076b0c55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"
Jul 28 19:44:09.631838 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.631788    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-maas-api-tls\") pod \"maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695\" (UID: \"bcac02cc-aa07-4c95-bd7b-6be5076b0c55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"
Jul 28 19:44:09.633332 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.633309    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"]
Jul 28 19:44:09.634502 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.634438    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-maas-api-tls\") pod \"maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695\" (UID: \"bcac02cc-aa07-4c95-bd7b-6be5076b0c55\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"
Jul 28 19:44:09.637130 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.637112    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"
Jul 28 19:44:09.639607 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.639589    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-d66111-dockercfg-j4n4l\""
Jul 28 19:44:09.640130 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.640116    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-d66111\""
Jul 28 19:44:09.648898 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.648876    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"]
Jul 28 19:44:09.732766 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.732741    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7923b780-8184-4612-b7ee-c480ab60cab3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv\" (UID: \"7923b780-8184-4612-b7ee-c480ab60cab3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"
Jul 28 19:44:09.732865 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.732795    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fcnj\" (UniqueName: \"kubernetes.io/projected/7923b780-8184-4612-b7ee-c480ab60cab3-kube-api-access-9fcnj\") pod \"payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv\" (UID: \"7923b780-8184-4612-b7ee-c480ab60cab3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"
Jul 28 19:44:09.782755 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.782730    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"
Jul 28 19:44:09.833506 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.833450    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fcnj\" (UniqueName: \"kubernetes.io/projected/7923b780-8184-4612-b7ee-c480ab60cab3-kube-api-access-9fcnj\") pod \"payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv\" (UID: \"7923b780-8184-4612-b7ee-c480ab60cab3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"
Jul 28 19:44:09.833611 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.833572    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7923b780-8184-4612-b7ee-c480ab60cab3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv\" (UID: \"7923b780-8184-4612-b7ee-c480ab60cab3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"
Jul 28 19:44:09.834553 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.834528    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7923b780-8184-4612-b7ee-c480ab60cab3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv\" (UID: \"7923b780-8184-4612-b7ee-c480ab60cab3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"
Jul 28 19:44:09.843368 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.843238    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fcnj\" (UniqueName: \"kubernetes.io/projected/7923b780-8184-4612-b7ee-c480ab60cab3-kube-api-access-9fcnj\") pod \"payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv\" (UID: \"7923b780-8184-4612-b7ee-c480ab60cab3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"
Jul 28 19:44:09.911137 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.911110    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"]
Jul 28 19:44:09.913116 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:44:09.913088    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcac02cc_aa07_4c95_bd7b_6be5076b0c55.slice/crio-1ea13b84c0a9b603447fe47f6d726c3ccf5dcd8dde05e8868f826e0659e4d4df WatchSource:0}: Error finding container 1ea13b84c0a9b603447fe47f6d726c3ccf5dcd8dde05e8868f826e0659e4d4df: Status 404 returned error can't find the container with id 1ea13b84c0a9b603447fe47f6d726c3ccf5dcd8dde05e8868f826e0659e4d4df
Jul 28 19:44:09.953751 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:09.953731    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"
Jul 28 19:44:10.084142 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:10.084117    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"]
Jul 28 19:44:10.085810 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:44:10.085786    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7923b780_8184_4612_b7ee_c480ab60cab3.slice/crio-a8ca2f3caf5e92a74896687c8110fa3b34925d78a65e52c9fd568827986be5cb WatchSource:0}: Error finding container a8ca2f3caf5e92a74896687c8110fa3b34925d78a65e52c9fd568827986be5cb: Status 404 returned error can't find the container with id a8ca2f3caf5e92a74896687c8110fa3b34925d78a65e52c9fd568827986be5cb
Jul 28 19:44:10.437924 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:10.437892    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv" event={"ID":"7923b780-8184-4612-b7ee-c480ab60cab3","Type":"ContainerStarted","Data":"ba699cf8101b011eeaefeb5bb5c80869cd6a1b236c62538288723d3d6736610a"}
Jul 28 19:44:10.438050 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:10.437955    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv" event={"ID":"7923b780-8184-4612-b7ee-c480ab60cab3","Type":"ContainerStarted","Data":"a8ca2f3caf5e92a74896687c8110fa3b34925d78a65e52c9fd568827986be5cb"}
Jul 28 19:44:10.438050 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:10.437984    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"
Jul 28 19:44:10.439692 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:10.439666    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695" event={"ID":"bcac02cc-aa07-4c95-bd7b-6be5076b0c55","Type":"ContainerStarted","Data":"d0148edd30ea89e507cef32c7dc728079fcb6a4659be1befe02a1281b13a976a"}
Jul 28 19:44:10.439802 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:10.439712    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"
Jul 28 19:44:10.439802 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:10.439732    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695" event={"ID":"bcac02cc-aa07-4c95-bd7b-6be5076b0c55","Type":"ContainerStarted","Data":"1ea13b84c0a9b603447fe47f6d726c3ccf5dcd8dde05e8868f826e0659e4d4df"}
Jul 28 19:44:10.461361 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:10.461321    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv" podStartSLOduration=1.461309798 podStartE2EDuration="1.461309798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:44:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:44:10.459657389 +0000 UTC m=+2498.406142257" watchObservedRunningTime="2026-07-28 19:44:10.461309798 +0000 UTC m=+2498.407794664"
Jul 28 19:44:10.482209 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:10.482164    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695" podStartSLOduration=2.081165002 podStartE2EDuration="2.482150987s" totalImagesPullingTime="400.985985ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:44:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:44:10.480946203 +0000 UTC m=+2498.427431095" watchObservedRunningTime="2026-07-28 19:44:10.482150987 +0000 UTC m=+2498.428635854"
Jul 28 19:44:16.449567 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:16.449532    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"
Jul 28 19:44:21.447118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:21.447086    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"
Jul 28 19:44:48.646448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.646409    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"]
Jul 28 19:44:48.650450 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.650427    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.654387 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.654365    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ec81e7\"/\"openshift-service-ca.crt\""
Jul 28 19:44:48.655246 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.655223    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ec81e7\"/\"auth-test-model-ec81e7-kserve-self-signed-certs\""
Jul 28 19:44:48.655413 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.655395    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ec81e7\"/\"kube-root-ca.crt\""
Jul 28 19:44:48.655515 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.655455    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ec81e7\"/\"default-dockercfg-442dt\""
Jul 28 19:44:48.667433 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.667411    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"]
Jul 28 19:44:48.753401 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.753373    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-dshm\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.753536 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.753411    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-tmp-dir\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.753536 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.753443    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-457pc\" (UniqueName: \"kubernetes.io/projected/362dca00-5158-4c2e-8a10-d80e235dcfe2-kube-api-access-457pc\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.753671 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.753545    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-home\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.753671 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.753570    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-model-cache\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.753671 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.753653    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/362dca00-5158-4c2e-8a10-d80e235dcfe2-tls-certs\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.854217 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.854190    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/362dca00-5158-4c2e-8a10-d80e235dcfe2-tls-certs\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.854311 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.854258    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-dshm\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.854311 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.854288    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-tmp-dir\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.854418 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.854333    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-457pc\" (UniqueName: \"kubernetes.io/projected/362dca00-5158-4c2e-8a10-d80e235dcfe2-kube-api-access-457pc\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.854418 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.854356    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-home\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.854540 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.854475    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-model-cache\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.854748 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.854728    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-home\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.854815 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.854754    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-tmp-dir\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.854865 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.854831    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-model-cache\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.856599 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.856576    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-dshm\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.856753 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.856735    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/362dca00-5158-4c2e-8a10-d80e235dcfe2-tls-certs\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.863477 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.863452    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-457pc\" (UniqueName: \"kubernetes.io/projected/362dca00-5158-4c2e-8a10-d80e235dcfe2-kube-api-access-457pc\") pod \"auth-test-model-ec81e7-kserve-7d574977bc-pnz7b\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") " pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:48.959826 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:48.959804    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:49.093982 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:49.093958    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"]
Jul 28 19:44:49.095888 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:44:49.095859    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod362dca00_5158_4c2e_8a10_d80e235dcfe2.slice/crio-179052cab17eb558cae642c7c053cf2ff382314f89d3907d682060dabe786e0e WatchSource:0}: Error finding container 179052cab17eb558cae642c7c053cf2ff382314f89d3907d682060dabe786e0e: Status 404 returned error can't find the container with id 179052cab17eb558cae642c7c053cf2ff382314f89d3907d682060dabe786e0e
Jul 28 19:44:49.630440 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:49.630407    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b" event={"ID":"362dca00-5158-4c2e-8a10-d80e235dcfe2","Type":"ContainerStarted","Data":"a6c2330ffea148ea2e3907efba6b65311a70878d302f2acce6e6013bf17ea949"}
Jul 28 19:44:49.630641 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:49.630447    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b" event={"ID":"362dca00-5158-4c2e-8a10-d80e235dcfe2","Type":"ContainerStarted","Data":"179052cab17eb558cae642c7c053cf2ff382314f89d3907d682060dabe786e0e"}
Jul 28 19:44:49.652780 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:49.652733    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b" podStartSLOduration=1.652716607 podStartE2EDuration="1.652716607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:44:49.650439336 +0000 UTC m=+2537.596924194" watchObservedRunningTime="2026-07-28 19:44:49.652716607 +0000 UTC m=+2537.599201475"
Jul 28 19:44:58.960110 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:58.960066    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:58.960110 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:58.960117    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:58.973009 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:58.972980    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:44:59.690431 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:44:59.690403    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:47:32.655096 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:47:32.655070    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:47:32.661246 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:47:32.661222    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:47:32.662264 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:47:32.662244    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:47:32.667693 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:47:32.667676    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:49:41.703416 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:41.703252    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"]
Jul 28 19:49:51.638492 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.638436    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"]
Jul 28 19:49:51.642130 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.642111    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.644848 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.644813    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ec81e7\"/\"sub-test-model-ec81e7-kserve-self-signed-certs\""
Jul 28 19:49:51.651605 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.651574    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"]
Jul 28 19:49:51.664294 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.664261    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-tmp-dir\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.664588 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.664559    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvrbt\" (UniqueName: \"kubernetes.io/projected/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-kube-api-access-rvrbt\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.665001 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.664626    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-dshm\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.665001 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.664704    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-model-cache\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.665001 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.664740    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-tls-certs\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.665001 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.664881    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-home\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.765947 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.765918    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvrbt\" (UniqueName: \"kubernetes.io/projected/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-kube-api-access-rvrbt\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.766052 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.765964    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-dshm\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.766052 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.765986    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-model-cache\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.766052 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.766003    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-tls-certs\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.766052 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.766036    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-home\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.766279 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.766084    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-tmp-dir\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.766470 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.766452    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-home\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.766580 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.766519    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-model-cache\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.766643 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.766576    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-tmp-dir\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.768288 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.768266    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-dshm\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.768762 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.768744    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-tls-certs\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.773807 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.773777    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvrbt\" (UniqueName: \"kubernetes.io/projected/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-kube-api-access-rvrbt\") pod \"sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") " pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:51.953462 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:51.953438    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:49:52.082836 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:52.082813    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"]
Jul 28 19:49:52.084549 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:49:52.084524    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5eddb54_c1ac_4a2b_851b_7f99078ce2f9.slice/crio-9c8088900441a625f7f55b56cdbae97b08423d7eaea02323f67b8c48afd554cc WatchSource:0}: Error finding container 9c8088900441a625f7f55b56cdbae97b08423d7eaea02323f67b8c48afd554cc: Status 404 returned error can't find the container with id 9c8088900441a625f7f55b56cdbae97b08423d7eaea02323f67b8c48afd554cc
Jul 28 19:49:52.107990 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:52.107963    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp" event={"ID":"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9","Type":"ContainerStarted","Data":"9c8088900441a625f7f55b56cdbae97b08423d7eaea02323f67b8c48afd554cc"}
Jul 28 19:49:53.115318 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:53.115279    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp" event={"ID":"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9","Type":"ContainerStarted","Data":"5922fc3b65be3c370a15a5fede9dde9191905d3d91976e18efe193c0806ca54a"}
Jul 28 19:49:53.131868 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:53.131808    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp" podStartSLOduration=2.131789204 podStartE2EDuration="2.131789204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:49:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:49:53.131400651 +0000 UTC m=+2841.077885518" watchObservedRunningTime="2026-07-28 19:49:53.131789204 +0000 UTC m=+2841.078274072"
Jul 28 19:49:56.722570 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:56.722531    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b" podUID="362dca00-5158-4c2e-8a10-d80e235dcfe2" containerName="main" containerID="cri-o://a6c2330ffea148ea2e3907efba6b65311a70878d302f2acce6e6013bf17ea949" gracePeriod=45
Jul 28 19:49:56.971948 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:56.971925    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:49:57.014334 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.014271    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-home\") pod \"362dca00-5158-4c2e-8a10-d80e235dcfe2\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") "
Jul 28 19:49:57.014334 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.014303    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-457pc\" (UniqueName: \"kubernetes.io/projected/362dca00-5158-4c2e-8a10-d80e235dcfe2-kube-api-access-457pc\") pod \"362dca00-5158-4c2e-8a10-d80e235dcfe2\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") "
Jul 28 19:49:57.014334 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.014329    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-model-cache\") pod \"362dca00-5158-4c2e-8a10-d80e235dcfe2\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") "
Jul 28 19:49:57.014601 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.014351    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/362dca00-5158-4c2e-8a10-d80e235dcfe2-tls-certs\") pod \"362dca00-5158-4c2e-8a10-d80e235dcfe2\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") "
Jul 28 19:49:57.014601 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.014385    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-tmp-dir\") pod \"362dca00-5158-4c2e-8a10-d80e235dcfe2\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") "
Jul 28 19:49:57.014601 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.014432    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-dshm\") pod \"362dca00-5158-4c2e-8a10-d80e235dcfe2\" (UID: \"362dca00-5158-4c2e-8a10-d80e235dcfe2\") "
Jul 28 19:49:57.014758 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.014615    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-home" (OuterVolumeSpecName: "home") pod "362dca00-5158-4c2e-8a10-d80e235dcfe2" (UID: "362dca00-5158-4c2e-8a10-d80e235dcfe2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:49:57.014758 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.014623    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-model-cache" (OuterVolumeSpecName: "model-cache") pod "362dca00-5158-4c2e-8a10-d80e235dcfe2" (UID: "362dca00-5158-4c2e-8a10-d80e235dcfe2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:49:57.014758 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.014665    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "362dca00-5158-4c2e-8a10-d80e235dcfe2" (UID: "362dca00-5158-4c2e-8a10-d80e235dcfe2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:49:57.014905 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.014890    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-home\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:49:57.014959 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.014907    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-model-cache\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:49:57.014959 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.014920    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-tmp-dir\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:49:57.016618 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.016595    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-dshm" (OuterVolumeSpecName: "dshm") pod "362dca00-5158-4c2e-8a10-d80e235dcfe2" (UID: "362dca00-5158-4c2e-8a10-d80e235dcfe2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:49:57.016719 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.016672    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/362dca00-5158-4c2e-8a10-d80e235dcfe2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "362dca00-5158-4c2e-8a10-d80e235dcfe2" (UID: "362dca00-5158-4c2e-8a10-d80e235dcfe2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:49:57.016845 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.016824    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/362dca00-5158-4c2e-8a10-d80e235dcfe2-kube-api-access-457pc" (OuterVolumeSpecName: "kube-api-access-457pc") pod "362dca00-5158-4c2e-8a10-d80e235dcfe2" (UID: "362dca00-5158-4c2e-8a10-d80e235dcfe2"). InnerVolumeSpecName "kube-api-access-457pc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:49:57.115627 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.115602    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/362dca00-5158-4c2e-8a10-d80e235dcfe2-tls-certs\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:49:57.115627 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.115624    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/362dca00-5158-4c2e-8a10-d80e235dcfe2-dshm\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:49:57.115752 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.115637    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-457pc\" (UniqueName: \"kubernetes.io/projected/362dca00-5158-4c2e-8a10-d80e235dcfe2-kube-api-access-457pc\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:49:57.134829 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.134800    2561 generic.go:356] "Generic (PLEG): container finished" podID="362dca00-5158-4c2e-8a10-d80e235dcfe2" containerID="a6c2330ffea148ea2e3907efba6b65311a70878d302f2acce6e6013bf17ea949" exitCode=0
Jul 28 19:49:57.134904 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.134858    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b" event={"ID":"362dca00-5158-4c2e-8a10-d80e235dcfe2","Type":"ContainerDied","Data":"a6c2330ffea148ea2e3907efba6b65311a70878d302f2acce6e6013bf17ea949"}
Jul 28 19:49:57.134954 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.134904    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"
Jul 28 19:49:57.134954 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.134907    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b" event={"ID":"362dca00-5158-4c2e-8a10-d80e235dcfe2","Type":"ContainerDied","Data":"179052cab17eb558cae642c7c053cf2ff382314f89d3907d682060dabe786e0e"}
Jul 28 19:49:57.134954 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.134934    2561 scope.go:117] "RemoveContainer" containerID="a6c2330ffea148ea2e3907efba6b65311a70878d302f2acce6e6013bf17ea949"
Jul 28 19:49:57.143460 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.143445    2561 scope.go:117] "RemoveContainer" containerID="a6c2330ffea148ea2e3907efba6b65311a70878d302f2acce6e6013bf17ea949"
Jul 28 19:49:57.143769 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:49:57.143749    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6c2330ffea148ea2e3907efba6b65311a70878d302f2acce6e6013bf17ea949\": container with ID starting with a6c2330ffea148ea2e3907efba6b65311a70878d302f2acce6e6013bf17ea949 not found: ID does not exist" containerID="a6c2330ffea148ea2e3907efba6b65311a70878d302f2acce6e6013bf17ea949"
Jul 28 19:49:57.143822 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.143779    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6c2330ffea148ea2e3907efba6b65311a70878d302f2acce6e6013bf17ea949"} err="failed to get container status \"a6c2330ffea148ea2e3907efba6b65311a70878d302f2acce6e6013bf17ea949\": rpc error: code = NotFound desc = could not find container \"a6c2330ffea148ea2e3907efba6b65311a70878d302f2acce6e6013bf17ea949\": container with ID starting with a6c2330ffea148ea2e3907efba6b65311a70878d302f2acce6e6013bf17ea949 not found: ID does not exist"
Jul 28 19:49:57.154394 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.154375    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"]
Jul 28 19:49:57.157265 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:57.157247    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/auth-test-model-ec81e7-kserve-7d574977bc-pnz7b"]
Jul 28 19:49:58.604496 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:49:58.604442    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="362dca00-5158-4c2e-8a10-d80e235dcfe2" path="/var/lib/kubelet/pods/362dca00-5158-4c2e-8a10-d80e235dcfe2/volumes"
Jul 28 19:50:01.953782 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:01.953753    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:50:01.954136 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:01.953804    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:50:01.965888 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:01.965863    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:50:02.178918 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:02.178898    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:50:21.545367 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.545319    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"]
Jul 28 19:50:21.546026 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.545957    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="362dca00-5158-4c2e-8a10-d80e235dcfe2" containerName="main"
Jul 28 19:50:21.546026 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.545976    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="362dca00-5158-4c2e-8a10-d80e235dcfe2" containerName="main"
Jul 28 19:50:21.546116 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.546036    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="362dca00-5158-4c2e-8a10-d80e235dcfe2" containerName="main"
Jul 28 19:50:21.549462 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.549442    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.553150 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.553110    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d66111\"/\"openshift-service-ca.crt\""
Jul 28 19:50:21.553150 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.553139    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d66111\"/\"sub-test-model-d66111-kserve-self-signed-certs\""
Jul 28 19:50:21.553373 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.553282    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d66111\"/\"kube-root-ca.crt\""
Jul 28 19:50:21.553440 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.553420    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d66111\"/\"default-dockercfg-4fkhj\""
Jul 28 19:50:21.559044 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.558996    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"]
Jul 28 19:50:21.621974 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.621950    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-dshm\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.622084 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.621986    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-home\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.622084 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.622009    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed50599c-46a4-48cf-88ac-860dac550ce1-tls-certs\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.622084 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.622055    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tlhm\" (UniqueName: \"kubernetes.io/projected/ed50599c-46a4-48cf-88ac-860dac550ce1-kube-api-access-4tlhm\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.622188 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.622108    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-model-cache\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.622188 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.622168    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-tmp-dir\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.722464 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.722436    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-dshm\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.722597 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.722559    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-home\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.722597 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.722593    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed50599c-46a4-48cf-88ac-860dac550ce1-tls-certs\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.722732 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.722619    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tlhm\" (UniqueName: \"kubernetes.io/projected/ed50599c-46a4-48cf-88ac-860dac550ce1-kube-api-access-4tlhm\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.722924 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.722900    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-home\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.723056 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.722907    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-model-cache\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.723194 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.723174    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-model-cache\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.723259 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.723205    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-tmp-dir\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.723640 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.723615    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-tmp-dir\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.724953 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.724935    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-dshm\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.725268 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.725252    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed50599c-46a4-48cf-88ac-860dac550ce1-tls-certs\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.730171 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.730151    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tlhm\" (UniqueName: \"kubernetes.io/projected/ed50599c-46a4-48cf-88ac-860dac550ce1-kube-api-access-4tlhm\") pod \"sub-test-model-d66111-kserve-f468c76cc-q5t6b\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") " pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.861886 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.861835    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:21.990183 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:21.990159    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"]
Jul 28 19:50:21.991812 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:50:21.991784    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded50599c_46a4_48cf_88ac_860dac550ce1.slice/crio-e71968947e02d959a77ae0e13c41ef9dd29895b0c0ea71ef8fcace27d849d70a WatchSource:0}: Error finding container e71968947e02d959a77ae0e13c41ef9dd29895b0c0ea71ef8fcace27d849d70a: Status 404 returned error can't find the container with id e71968947e02d959a77ae0e13c41ef9dd29895b0c0ea71ef8fcace27d849d70a
Jul 28 19:50:22.268287 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:22.268259    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b" event={"ID":"ed50599c-46a4-48cf-88ac-860dac550ce1","Type":"ContainerStarted","Data":"58262f9c9bb355b02d27af15c28fe22dbf145ade3ca120a6127b27d0cf2e16af"}
Jul 28 19:50:22.268407 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:22.268294    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b" event={"ID":"ed50599c-46a4-48cf-88ac-860dac550ce1","Type":"ContainerStarted","Data":"e71968947e02d959a77ae0e13c41ef9dd29895b0c0ea71ef8fcace27d849d70a"}
Jul 28 19:50:22.288088 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:22.288042    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b" podStartSLOduration=1.288023814 podStartE2EDuration="1.288023814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:50:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:50:22.286117306 +0000 UTC m=+2870.232602172" watchObservedRunningTime="2026-07-28 19:50:22.288023814 +0000 UTC m=+2870.234508683"
Jul 28 19:50:31.862957 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:31.862861    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:31.862957 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:31.862921    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:31.874935 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:31.874904    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:50:32.339107 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:50:32.339080    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:51:27.353113 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:27.353075    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"]
Jul 28 19:51:28.306542 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:28.306508    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"]
Jul 28 19:51:42.371756 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.371712    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp" podUID="b5eddb54-c1ac-4a2b-851b-7f99078ce2f9" containerName="main" containerID="cri-o://5922fc3b65be3c370a15a5fede9dde9191905d3d91976e18efe193c0806ca54a" gracePeriod=45
Jul 28 19:51:42.618369 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.618344    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:51:42.683819 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.683749    2561 generic.go:356] "Generic (PLEG): container finished" podID="b5eddb54-c1ac-4a2b-851b-7f99078ce2f9" containerID="5922fc3b65be3c370a15a5fede9dde9191905d3d91976e18efe193c0806ca54a" exitCode=0
Jul 28 19:51:42.683819 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.683782    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"
Jul 28 19:51:42.683819 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.683799    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp" event={"ID":"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9","Type":"ContainerDied","Data":"5922fc3b65be3c370a15a5fede9dde9191905d3d91976e18efe193c0806ca54a"}
Jul 28 19:51:42.684002 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.683837    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp" event={"ID":"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9","Type":"ContainerDied","Data":"9c8088900441a625f7f55b56cdbae97b08423d7eaea02323f67b8c48afd554cc"}
Jul 28 19:51:42.684002 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.683857    2561 scope.go:117] "RemoveContainer" containerID="5922fc3b65be3c370a15a5fede9dde9191905d3d91976e18efe193c0806ca54a"
Jul 28 19:51:42.691911 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.691897    2561 scope.go:117] "RemoveContainer" containerID="5922fc3b65be3c370a15a5fede9dde9191905d3d91976e18efe193c0806ca54a"
Jul 28 19:51:42.692173 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:51:42.692155    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5922fc3b65be3c370a15a5fede9dde9191905d3d91976e18efe193c0806ca54a\": container with ID starting with 5922fc3b65be3c370a15a5fede9dde9191905d3d91976e18efe193c0806ca54a not found: ID does not exist" containerID="5922fc3b65be3c370a15a5fede9dde9191905d3d91976e18efe193c0806ca54a"
Jul 28 19:51:42.692219 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.692181    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5922fc3b65be3c370a15a5fede9dde9191905d3d91976e18efe193c0806ca54a"} err="failed to get container status \"5922fc3b65be3c370a15a5fede9dde9191905d3d91976e18efe193c0806ca54a\": rpc error: code = NotFound desc = could not find container \"5922fc3b65be3c370a15a5fede9dde9191905d3d91976e18efe193c0806ca54a\": container with ID starting with 5922fc3b65be3c370a15a5fede9dde9191905d3d91976e18efe193c0806ca54a not found: ID does not exist"
Jul 28 19:51:42.765681 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.765659    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-home\") pod \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") "
Jul 28 19:51:42.765779 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.765691    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvrbt\" (UniqueName: \"kubernetes.io/projected/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-kube-api-access-rvrbt\") pod \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") "
Jul 28 19:51:42.765779 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.765716    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-tls-certs\") pod \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") "
Jul 28 19:51:42.765779 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.765755    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-model-cache\") pod \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") "
Jul 28 19:51:42.765895 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.765832    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-dshm\") pod \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") "
Jul 28 19:51:42.765895 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.765869    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-tmp-dir\") pod \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\" (UID: \"b5eddb54-c1ac-4a2b-851b-7f99078ce2f9\") "
Jul 28 19:51:42.765997 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.765928    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-home" (OuterVolumeSpecName: "home") pod "b5eddb54-c1ac-4a2b-851b-7f99078ce2f9" (UID: "b5eddb54-c1ac-4a2b-851b-7f99078ce2f9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:51:42.766088 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.766064    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-model-cache" (OuterVolumeSpecName: "model-cache") pod "b5eddb54-c1ac-4a2b-851b-7f99078ce2f9" (UID: "b5eddb54-c1ac-4a2b-851b-7f99078ce2f9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:51:42.766180 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.766162    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b5eddb54-c1ac-4a2b-851b-7f99078ce2f9" (UID: "b5eddb54-c1ac-4a2b-851b-7f99078ce2f9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:51:42.766256 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.766237    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-home\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:51:42.766295 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.766263    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-model-cache\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:51:42.766328 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.766297    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-tmp-dir\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:51:42.767739 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.767714    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-kube-api-access-rvrbt" (OuterVolumeSpecName: "kube-api-access-rvrbt") pod "b5eddb54-c1ac-4a2b-851b-7f99078ce2f9" (UID: "b5eddb54-c1ac-4a2b-851b-7f99078ce2f9"). InnerVolumeSpecName "kube-api-access-rvrbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:51:42.767821 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.767740    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b5eddb54-c1ac-4a2b-851b-7f99078ce2f9" (UID: "b5eddb54-c1ac-4a2b-851b-7f99078ce2f9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:51:42.767864 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.767818    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-dshm" (OuterVolumeSpecName: "dshm") pod "b5eddb54-c1ac-4a2b-851b-7f99078ce2f9" (UID: "b5eddb54-c1ac-4a2b-851b-7f99078ce2f9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:51:42.867462 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.867443    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-dshm\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:51:42.867462 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.867463    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvrbt\" (UniqueName: \"kubernetes.io/projected/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-kube-api-access-rvrbt\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:51:42.867604 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:42.867473    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9-tls-certs\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:51:43.007181 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.007160    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"]
Jul 28 19:51:43.008895 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.008878    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ec81e7/sub-test-model-ec81e7-kserve-6d746b79d9-t8vmp"]
Jul 28 19:51:43.325048 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.324976    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b" podUID="ed50599c-46a4-48cf-88ac-860dac550ce1" containerName="main" containerID="cri-o://58262f9c9bb355b02d27af15c28fe22dbf145ade3ca120a6127b27d0cf2e16af" gracePeriod=45
Jul 28 19:51:43.576760 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.576701    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:51:43.673217 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.673194    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tlhm\" (UniqueName: \"kubernetes.io/projected/ed50599c-46a4-48cf-88ac-860dac550ce1-kube-api-access-4tlhm\") pod \"ed50599c-46a4-48cf-88ac-860dac550ce1\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") "
Jul 28 19:51:43.673326 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.673234    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed50599c-46a4-48cf-88ac-860dac550ce1-tls-certs\") pod \"ed50599c-46a4-48cf-88ac-860dac550ce1\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") "
Jul 28 19:51:43.673326 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.673305    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-model-cache\") pod \"ed50599c-46a4-48cf-88ac-860dac550ce1\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") "
Jul 28 19:51:43.673413 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.673328    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-tmp-dir\") pod \"ed50599c-46a4-48cf-88ac-860dac550ce1\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") "
Jul 28 19:51:43.673413 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.673352    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-home\") pod \"ed50599c-46a4-48cf-88ac-860dac550ce1\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") "
Jul 28 19:51:43.673413 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.673380    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-dshm\") pod \"ed50599c-46a4-48cf-88ac-860dac550ce1\" (UID: \"ed50599c-46a4-48cf-88ac-860dac550ce1\") "
Jul 28 19:51:43.673643 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.673603    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ed50599c-46a4-48cf-88ac-860dac550ce1" (UID: "ed50599c-46a4-48cf-88ac-860dac550ce1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:51:43.673737 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.673657    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-home" (OuterVolumeSpecName: "home") pod "ed50599c-46a4-48cf-88ac-860dac550ce1" (UID: "ed50599c-46a4-48cf-88ac-860dac550ce1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:51:43.673737 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.673596    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-model-cache" (OuterVolumeSpecName: "model-cache") pod "ed50599c-46a4-48cf-88ac-860dac550ce1" (UID: "ed50599c-46a4-48cf-88ac-860dac550ce1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:51:43.675315 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.675288    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-dshm" (OuterVolumeSpecName: "dshm") pod "ed50599c-46a4-48cf-88ac-860dac550ce1" (UID: "ed50599c-46a4-48cf-88ac-860dac550ce1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:51:43.675411 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.675351    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed50599c-46a4-48cf-88ac-860dac550ce1-kube-api-access-4tlhm" (OuterVolumeSpecName: "kube-api-access-4tlhm") pod "ed50599c-46a4-48cf-88ac-860dac550ce1" (UID: "ed50599c-46a4-48cf-88ac-860dac550ce1"). InnerVolumeSpecName "kube-api-access-4tlhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:51:43.675472 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.675411    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed50599c-46a4-48cf-88ac-860dac550ce1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ed50599c-46a4-48cf-88ac-860dac550ce1" (UID: "ed50599c-46a4-48cf-88ac-860dac550ce1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:51:43.691085 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.691062    2561 generic.go:356] "Generic (PLEG): container finished" podID="ed50599c-46a4-48cf-88ac-860dac550ce1" containerID="58262f9c9bb355b02d27af15c28fe22dbf145ade3ca120a6127b27d0cf2e16af" exitCode=0
Jul 28 19:51:43.691183 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.691104    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"
Jul 28 19:51:43.691183 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.691120    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b" event={"ID":"ed50599c-46a4-48cf-88ac-860dac550ce1","Type":"ContainerDied","Data":"58262f9c9bb355b02d27af15c28fe22dbf145ade3ca120a6127b27d0cf2e16af"}
Jul 28 19:51:43.691183 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.691150    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b" event={"ID":"ed50599c-46a4-48cf-88ac-860dac550ce1","Type":"ContainerDied","Data":"e71968947e02d959a77ae0e13c41ef9dd29895b0c0ea71ef8fcace27d849d70a"}
Jul 28 19:51:43.691183 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.691167    2561 scope.go:117] "RemoveContainer" containerID="58262f9c9bb355b02d27af15c28fe22dbf145ade3ca120a6127b27d0cf2e16af"
Jul 28 19:51:43.699165 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.699151    2561 scope.go:117] "RemoveContainer" containerID="58262f9c9bb355b02d27af15c28fe22dbf145ade3ca120a6127b27d0cf2e16af"
Jul 28 19:51:43.699432 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:51:43.699416    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58262f9c9bb355b02d27af15c28fe22dbf145ade3ca120a6127b27d0cf2e16af\": container with ID starting with 58262f9c9bb355b02d27af15c28fe22dbf145ade3ca120a6127b27d0cf2e16af not found: ID does not exist" containerID="58262f9c9bb355b02d27af15c28fe22dbf145ade3ca120a6127b27d0cf2e16af"
Jul 28 19:51:43.699509 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.699442    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58262f9c9bb355b02d27af15c28fe22dbf145ade3ca120a6127b27d0cf2e16af"} err="failed to get container status \"58262f9c9bb355b02d27af15c28fe22dbf145ade3ca120a6127b27d0cf2e16af\": rpc error: code = NotFound desc = could not find container \"58262f9c9bb355b02d27af15c28fe22dbf145ade3ca120a6127b27d0cf2e16af\": container with ID starting with 58262f9c9bb355b02d27af15c28fe22dbf145ade3ca120a6127b27d0cf2e16af not found: ID does not exist"
Jul 28 19:51:43.712680 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.712659    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"]
Jul 28 19:51:43.717336 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.717315    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d66111/sub-test-model-d66111-kserve-f468c76cc-q5t6b"]
Jul 28 19:51:43.774908 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.774887    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tlhm\" (UniqueName: \"kubernetes.io/projected/ed50599c-46a4-48cf-88ac-860dac550ce1-kube-api-access-4tlhm\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:51:43.774989 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.774908    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed50599c-46a4-48cf-88ac-860dac550ce1-tls-certs\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:51:43.774989 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.774919    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-model-cache\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:51:43.774989 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.774927    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-tmp-dir\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:51:43.774989 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.774935    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-home\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:51:43.774989 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:43.774941    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed50599c-46a4-48cf-88ac-860dac550ce1-dshm\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:51:44.604048 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:44.604025    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5eddb54-c1ac-4a2b-851b-7f99078ce2f9" path="/var/lib/kubelet/pods/b5eddb54-c1ac-4a2b-851b-7f99078ce2f9/volumes"
Jul 28 19:51:44.604440 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:44.604426    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed50599c-46a4-48cf-88ac-860dac550ce1" path="/var/lib/kubelet/pods/ed50599c-46a4-48cf-88ac-860dac550ce1/volumes"
Jul 28 19:51:51.339103 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.339073    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"]
Jul 28 19:51:51.339590 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.339412    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5eddb54-c1ac-4a2b-851b-7f99078ce2f9" containerName="main"
Jul 28 19:51:51.339590 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.339422    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5eddb54-c1ac-4a2b-851b-7f99078ce2f9" containerName="main"
Jul 28 19:51:51.339590 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.339440    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed50599c-46a4-48cf-88ac-860dac550ce1" containerName="main"
Jul 28 19:51:51.339590 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.339445    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed50599c-46a4-48cf-88ac-860dac550ce1" containerName="main"
Jul 28 19:51:51.339590 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.339515    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed50599c-46a4-48cf-88ac-860dac550ce1" containerName="main"
Jul 28 19:51:51.339590 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.339525    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5eddb54-c1ac-4a2b-851b-7f99078ce2f9" containerName="main"
Jul 28 19:51:51.343876 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.343858    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.347202 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.347180    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d66111\"/\"default-dockercfg-4fkhj\""
Jul 28 19:51:51.347202 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.347192    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d66111\"/\"openshift-service-ca.crt\""
Jul 28 19:51:51.347375 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.347184    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d66111\"/\"rate-test-model-d66111-kserve-self-signed-certs\""
Jul 28 19:51:51.347375 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.347222    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d66111\"/\"kube-root-ca.crt\""
Jul 28 19:51:51.352573 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.352552    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"]
Jul 28 19:51:51.435244 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.435223    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-model-cache\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.435348 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.435249    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-dshm\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.435348 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.435270    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-tmp-dir\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.435348 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.435291    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-home\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.435348 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.435321    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65q9h\" (UniqueName: \"kubernetes.io/projected/a448b21c-5d40-4a63-bc74-8348b1b0ae22-kube-api-access-65q9h\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.435561 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.435376    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a448b21c-5d40-4a63-bc74-8348b1b0ae22-tls-certs\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.536103 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.536081    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-model-cache\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.536204 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.536116    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-dshm\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.536204 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.536137    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-tmp-dir\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.536204 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.536153    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-home\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.536204 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.536167    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65q9h\" (UniqueName: \"kubernetes.io/projected/a448b21c-5d40-4a63-bc74-8348b1b0ae22-kube-api-access-65q9h\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.536392 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.536227    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a448b21c-5d40-4a63-bc74-8348b1b0ae22-tls-certs\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.536567 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.536543    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-model-cache\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.536567 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.536563    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-home\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.536681 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.536600    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-tmp-dir\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.538354 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.538338    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-dshm\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.538645 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.538628    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a448b21c-5d40-4a63-bc74-8348b1b0ae22-tls-certs\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.544950 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.544929    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65q9h\" (UniqueName: \"kubernetes.io/projected/a448b21c-5d40-4a63-bc74-8348b1b0ae22-kube-api-access-65q9h\") pod \"rate-test-model-d66111-kserve-799fcf867f-cc57b\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") " pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.652811 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.652765    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:51:51.777957 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:51.777933    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"]
Jul 28 19:51:51.779746 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:51:51.779723    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda448b21c_5d40_4a63_bc74_8348b1b0ae22.slice/crio-8b883744d58d858d5ad96e426be381aff2aa15d44c5562e3ac339e23dcd3b98f WatchSource:0}: Error finding container 8b883744d58d858d5ad96e426be381aff2aa15d44c5562e3ac339e23dcd3b98f: Status 404 returned error can't find the container with id 8b883744d58d858d5ad96e426be381aff2aa15d44c5562e3ac339e23dcd3b98f
Jul 28 19:51:52.737759 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:52.737723    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b" event={"ID":"a448b21c-5d40-4a63-bc74-8348b1b0ae22","Type":"ContainerStarted","Data":"4157e1a6eed469857755d140ec7e194173d671f8d348c9631066d45d3329bd06"}
Jul 28 19:51:52.738237 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:52.737771    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b" event={"ID":"a448b21c-5d40-4a63-bc74-8348b1b0ae22","Type":"ContainerStarted","Data":"8b883744d58d858d5ad96e426be381aff2aa15d44c5562e3ac339e23dcd3b98f"}
Jul 28 19:51:52.757000 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:51:52.756954    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b" podStartSLOduration=1.756933509 podStartE2EDuration="1.756933509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:51:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:51:52.754685661 +0000 UTC m=+2960.701170526" watchObservedRunningTime="2026-07-28 19:51:52.756933509 +0000 UTC m=+2960.703418375"
Jul 28 19:52:01.653467 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:52:01.653371    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:52:01.653467 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:52:01.653427    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:52:01.665531 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:52:01.665505    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:52:01.791613 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:52:01.791587    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:52:32.688149 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:52:32.688106    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:52:32.694469 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:52:32.694447    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:52:32.695204 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:52:32.695181    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:52:32.700713 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:52:32.700695    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:53:16.957891 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:16.957854    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"]
Jul 28 19:53:31.976399 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:31.976304    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b" podUID="a448b21c-5d40-4a63-bc74-8348b1b0ae22" containerName="main" containerID="cri-o://4157e1a6eed469857755d140ec7e194173d671f8d348c9631066d45d3329bd06" gracePeriod=45
Jul 28 19:53:32.215178 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.215153    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:53:32.215368 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.215348    2561 generic.go:356] "Generic (PLEG): container finished" podID="a448b21c-5d40-4a63-bc74-8348b1b0ae22" containerID="4157e1a6eed469857755d140ec7e194173d671f8d348c9631066d45d3329bd06" exitCode=0
Jul 28 19:53:32.215430 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.215408    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b" event={"ID":"a448b21c-5d40-4a63-bc74-8348b1b0ae22","Type":"ContainerDied","Data":"4157e1a6eed469857755d140ec7e194173d671f8d348c9631066d45d3329bd06"}
Jul 28 19:53:32.215471 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.215451    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b" event={"ID":"a448b21c-5d40-4a63-bc74-8348b1b0ae22","Type":"ContainerDied","Data":"8b883744d58d858d5ad96e426be381aff2aa15d44c5562e3ac339e23dcd3b98f"}
Jul 28 19:53:32.215471 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.215468    2561 scope.go:117] "RemoveContainer" containerID="4157e1a6eed469857755d140ec7e194173d671f8d348c9631066d45d3329bd06"
Jul 28 19:53:32.223354 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.223339    2561 scope.go:117] "RemoveContainer" containerID="4157e1a6eed469857755d140ec7e194173d671f8d348c9631066d45d3329bd06"
Jul 28 19:53:32.223636 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:53:32.223618    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4157e1a6eed469857755d140ec7e194173d671f8d348c9631066d45d3329bd06\": container with ID starting with 4157e1a6eed469857755d140ec7e194173d671f8d348c9631066d45d3329bd06 not found: ID does not exist" containerID="4157e1a6eed469857755d140ec7e194173d671f8d348c9631066d45d3329bd06"
Jul 28 19:53:32.223702 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.223649    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4157e1a6eed469857755d140ec7e194173d671f8d348c9631066d45d3329bd06"} err="failed to get container status \"4157e1a6eed469857755d140ec7e194173d671f8d348c9631066d45d3329bd06\": rpc error: code = NotFound desc = could not find container \"4157e1a6eed469857755d140ec7e194173d671f8d348c9631066d45d3329bd06\": container with ID starting with 4157e1a6eed469857755d140ec7e194173d671f8d348c9631066d45d3329bd06 not found: ID does not exist"
Jul 28 19:53:32.363526 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.363452    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-home\") pod \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") "
Jul 28 19:53:32.363526 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.363518    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a448b21c-5d40-4a63-bc74-8348b1b0ae22-tls-certs\") pod \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") "
Jul 28 19:53:32.363685 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.363544    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-dshm\") pod \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") "
Jul 28 19:53:32.363685 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.363577    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65q9h\" (UniqueName: \"kubernetes.io/projected/a448b21c-5d40-4a63-bc74-8348b1b0ae22-kube-api-access-65q9h\") pod \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") "
Jul 28 19:53:32.363685 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.363629    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-tmp-dir\") pod \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") "
Jul 28 19:53:32.363685 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.363673    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-model-cache\") pod \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\" (UID: \"a448b21c-5d40-4a63-bc74-8348b1b0ae22\") "
Jul 28 19:53:32.363885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.363720    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-home" (OuterVolumeSpecName: "home") pod "a448b21c-5d40-4a63-bc74-8348b1b0ae22" (UID: "a448b21c-5d40-4a63-bc74-8348b1b0ae22"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:53:32.364001 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.363959    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a448b21c-5d40-4a63-bc74-8348b1b0ae22" (UID: "a448b21c-5d40-4a63-bc74-8348b1b0ae22"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:53:32.364088 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.363975    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-model-cache" (OuterVolumeSpecName: "model-cache") pod "a448b21c-5d40-4a63-bc74-8348b1b0ae22" (UID: "a448b21c-5d40-4a63-bc74-8348b1b0ae22"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:53:32.364088 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.364019    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-home\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:53:32.365667 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.365645    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-dshm" (OuterVolumeSpecName: "dshm") pod "a448b21c-5d40-4a63-bc74-8348b1b0ae22" (UID: "a448b21c-5d40-4a63-bc74-8348b1b0ae22"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:53:32.365755 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.365704    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a448b21c-5d40-4a63-bc74-8348b1b0ae22-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a448b21c-5d40-4a63-bc74-8348b1b0ae22" (UID: "a448b21c-5d40-4a63-bc74-8348b1b0ae22"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:53:32.365755 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.365708    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a448b21c-5d40-4a63-bc74-8348b1b0ae22-kube-api-access-65q9h" (OuterVolumeSpecName: "kube-api-access-65q9h") pod "a448b21c-5d40-4a63-bc74-8348b1b0ae22" (UID: "a448b21c-5d40-4a63-bc74-8348b1b0ae22"). InnerVolumeSpecName "kube-api-access-65q9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:53:32.465000 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.464977    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a448b21c-5d40-4a63-bc74-8348b1b0ae22-tls-certs\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:53:32.465000 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.464997    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-dshm\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:53:32.465118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.465006    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65q9h\" (UniqueName: \"kubernetes.io/projected/a448b21c-5d40-4a63-bc74-8348b1b0ae22-kube-api-access-65q9h\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:53:32.465118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.465030    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-tmp-dir\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:53:32.465118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:32.465039    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a448b21c-5d40-4a63-bc74-8348b1b0ae22-model-cache\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:53:33.219654 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:33.219630    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"
Jul 28 19:53:33.251007 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:33.250980    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"]
Jul 28 19:53:33.254902 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:33.254876    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d66111/rate-test-model-d66111-kserve-799fcf867f-cc57b"]
Jul 28 19:53:34.603958 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:34.603923    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a448b21c-5d40-4a63-bc74-8348b1b0ae22" path="/var/lib/kubelet/pods/a448b21c-5d40-4a63-bc74-8348b1b0ae22/volumes"
Jul 28 19:53:40.450747 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:40.450712    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"]
Jul 28 19:53:40.451113 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:40.451063    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a448b21c-5d40-4a63-bc74-8348b1b0ae22" containerName="main"
Jul 28 19:53:40.451113 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:40.451074    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a448b21c-5d40-4a63-bc74-8348b1b0ae22" containerName="main"
Jul 28 19:53:40.451198 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:40.451148    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a448b21c-5d40-4a63-bc74-8348b1b0ae22" containerName="main"
Jul 28 19:53:40.455834 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:40.455814    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"
Jul 28 19:53:40.461769 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:40.461742    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-cf1393-serving-cert\""
Jul 28 19:53:40.471708 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:40.471687    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"]
Jul 28 19:53:40.526564 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:40.526537    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dee60a2-ae6f-4126-acae-05b0ead764c0-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h\" (UID: \"7dee60a2-ae6f-4126-acae-05b0ead764c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"
Jul 28 19:53:40.526661 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:40.526614    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4t9d\" (UniqueName: \"kubernetes.io/projected/7dee60a2-ae6f-4126-acae-05b0ead764c0-kube-api-access-m4t9d\") pod \"maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h\" (UID: \"7dee60a2-ae6f-4126-acae-05b0ead764c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"
Jul 28 19:53:40.526709 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:40.526660    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dee60a2-ae6f-4126-acae-05b0ead764c0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h\" (UID: \"7dee60a2-ae6f-4126-acae-05b0ead764c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"
Jul 28 19:53:40.627303 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:40.627269    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dee60a2-ae6f-4126-acae-05b0ead764c0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h\" (UID: \"7dee60a2-ae6f-4126-acae-05b0ead764c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"
Jul 28 19:53:40.627423 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:40.627327    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dee60a2-ae6f-4126-acae-05b0ead764c0-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h\" (UID: \"7dee60a2-ae6f-4126-acae-05b0ead764c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"
Jul 28 19:53:40.627423 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:40.627399    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4t9d\" (UniqueName: \"kubernetes.io/projected/7dee60a2-ae6f-4126-acae-05b0ead764c0-kube-api-access-m4t9d\") pod \"maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h\" (UID: \"7dee60a2-ae6f-4126-acae-05b0ead764c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"
Jul 28 19:53:40.627423 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:53:40.627411    2561 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-serving-cert: secret "maas-api-e2e-ipp-a-cf1393-serving-cert" not found
Jul 28 19:53:40.627557 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:53:40.627500    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dee60a2-ae6f-4126-acae-05b0ead764c0-maas-api-tls podName:7dee60a2-ae6f-4126-acae-05b0ead764c0 nodeName:}" failed. No retries permitted until 2026-07-28 19:53:41.127458576 +0000 UTC m=+3069.073943433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7dee60a2-ae6f-4126-acae-05b0ead764c0-maas-api-tls") pod "maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h" (UID: "7dee60a2-ae6f-4126-acae-05b0ead764c0") : secret "maas-api-e2e-ipp-a-cf1393-serving-cert" not found
Jul 28 19:53:40.627939 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:40.627921    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dee60a2-ae6f-4126-acae-05b0ead764c0-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h\" (UID: \"7dee60a2-ae6f-4126-acae-05b0ead764c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"
Jul 28 19:53:40.636683 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:40.636658    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4t9d\" (UniqueName: \"kubernetes.io/projected/7dee60a2-ae6f-4126-acae-05b0ead764c0-kube-api-access-m4t9d\") pod \"maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h\" (UID: \"7dee60a2-ae6f-4126-acae-05b0ead764c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"
Jul 28 19:53:41.130629 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.130602    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dee60a2-ae6f-4126-acae-05b0ead764c0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h\" (UID: \"7dee60a2-ae6f-4126-acae-05b0ead764c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"
Jul 28 19:53:41.132756 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.132739    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dee60a2-ae6f-4126-acae-05b0ead764c0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h\" (UID: \"7dee60a2-ae6f-4126-acae-05b0ead764c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"
Jul 28 19:53:41.234550 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.234522    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"]
Jul 28 19:53:41.238060 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.238045    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"
Jul 28 19:53:41.246642 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.246623    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-cf1393\""
Jul 28 19:53:41.257422 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.257398    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"]
Jul 28 19:53:41.288803 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.288780    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"]
Jul 28 19:53:41.292770 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.292750    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"
Jul 28 19:53:41.296417 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.296396    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-cf1393-dockercfg-dz7gt\""
Jul 28 19:53:41.301557 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.301536    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"]
Jul 28 19:53:41.332627 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.332603    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xclm\" (UniqueName: \"kubernetes.io/projected/93579aec-c805-4eda-afdd-77830b1f5fa0-kube-api-access-8xclm\") pod \"payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q\" (UID: \"93579aec-c805-4eda-afdd-77830b1f5fa0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"
Jul 28 19:53:41.332706 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.332687    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93579aec-c805-4eda-afdd-77830b1f5fa0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q\" (UID: \"93579aec-c805-4eda-afdd-77830b1f5fa0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"
Jul 28 19:53:41.365780 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.365753    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"
Jul 28 19:53:41.433512 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.433263    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldl9t\" (UniqueName: \"kubernetes.io/projected/e5025cdb-ab0d-457c-a948-11263ee3c692-kube-api-access-ldl9t\") pod \"payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj\" (UID: \"e5025cdb-ab0d-457c-a948-11263ee3c692\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"
Jul 28 19:53:41.433512 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.433321    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93579aec-c805-4eda-afdd-77830b1f5fa0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q\" (UID: \"93579aec-c805-4eda-afdd-77830b1f5fa0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"
Jul 28 19:53:41.433512 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.433388    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xclm\" (UniqueName: \"kubernetes.io/projected/93579aec-c805-4eda-afdd-77830b1f5fa0-kube-api-access-8xclm\") pod \"payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q\" (UID: \"93579aec-c805-4eda-afdd-77830b1f5fa0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"
Jul 28 19:53:41.433512 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.433504    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5025cdb-ab0d-457c-a948-11263ee3c692-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj\" (UID: \"e5025cdb-ab0d-457c-a948-11263ee3c692\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"
Jul 28 19:53:41.434124 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.434104    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93579aec-c805-4eda-afdd-77830b1f5fa0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q\" (UID: \"93579aec-c805-4eda-afdd-77830b1f5fa0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"
Jul 28 19:53:41.450311 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.450288    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xclm\" (UniqueName: \"kubernetes.io/projected/93579aec-c805-4eda-afdd-77830b1f5fa0-kube-api-access-8xclm\") pod \"payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q\" (UID: \"93579aec-c805-4eda-afdd-77830b1f5fa0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"
Jul 28 19:53:41.497866 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.497845    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"]
Jul 28 19:53:41.499857 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:53:41.499834    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dee60a2_ae6f_4126_acae_05b0ead764c0.slice/crio-f13ba933142ca59e5c3d2a9d5bf7acb39eefdeb15756163631e9aa4e0e15d9a1 WatchSource:0}: Error finding container f13ba933142ca59e5c3d2a9d5bf7acb39eefdeb15756163631e9aa4e0e15d9a1: Status 404 returned error can't find the container with id f13ba933142ca59e5c3d2a9d5bf7acb39eefdeb15756163631e9aa4e0e15d9a1
Jul 28 19:53:41.501104 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.501087    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:53:41.534202 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.534176    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5025cdb-ab0d-457c-a948-11263ee3c692-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj\" (UID: \"e5025cdb-ab0d-457c-a948-11263ee3c692\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"
Jul 28 19:53:41.534304 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.534220    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldl9t\" (UniqueName: \"kubernetes.io/projected/e5025cdb-ab0d-457c-a948-11263ee3c692-kube-api-access-ldl9t\") pod \"payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj\" (UID: \"e5025cdb-ab0d-457c-a948-11263ee3c692\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"
Jul 28 19:53:41.534808 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.534791    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5025cdb-ab0d-457c-a948-11263ee3c692-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj\" (UID: \"e5025cdb-ab0d-457c-a948-11263ee3c692\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"
Jul 28 19:53:41.541719 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.541703    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldl9t\" (UniqueName: \"kubernetes.io/projected/e5025cdb-ab0d-457c-a948-11263ee3c692-kube-api-access-ldl9t\") pod \"payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj\" (UID: \"e5025cdb-ab0d-457c-a948-11263ee3c692\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"
Jul 28 19:53:41.547540 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.547523    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"
Jul 28 19:53:41.605136 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.605112    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"
Jul 28 19:53:41.682671 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.682637    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"]
Jul 28 19:53:41.685235 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:53:41.685204    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93579aec_c805_4eda_afdd_77830b1f5fa0.slice/crio-e218f023006ace48efb58aba38515fa0962d475ab7bd97287a146013c932db36 WatchSource:0}: Error finding container e218f023006ace48efb58aba38515fa0962d475ab7bd97287a146013c932db36: Status 404 returned error can't find the container with id e218f023006ace48efb58aba38515fa0962d475ab7bd97287a146013c932db36
Jul 28 19:53:41.742541 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:41.742519    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"]
Jul 28 19:53:41.744970 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:53:41.744935    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5025cdb_ab0d_457c_a948_11263ee3c692.slice/crio-a8ecfa886137797361f2946444e8e46189ab0125f409b5823ddfccaea5e375b1 WatchSource:0}: Error finding container a8ecfa886137797361f2946444e8e46189ab0125f409b5823ddfccaea5e375b1: Status 404 returned error can't find the container with id a8ecfa886137797361f2946444e8e46189ab0125f409b5823ddfccaea5e375b1
Jul 28 19:53:42.268135 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:42.268041    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q" event={"ID":"93579aec-c805-4eda-afdd-77830b1f5fa0","Type":"ContainerStarted","Data":"b92f225b91511297af3dc0526a4c058db59085ef3d409e6128904173ca7859ed"}
Jul 28 19:53:42.268330 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:42.268212    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"
Jul 28 19:53:42.268330 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:42.268255    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q" event={"ID":"93579aec-c805-4eda-afdd-77830b1f5fa0","Type":"ContainerStarted","Data":"e218f023006ace48efb58aba38515fa0962d475ab7bd97287a146013c932db36"}
Jul 28 19:53:42.269444 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:42.269413    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj" event={"ID":"e5025cdb-ab0d-457c-a948-11263ee3c692","Type":"ContainerStarted","Data":"565aba29fe1f4ee9d01656c33a36d7f9b654911f6d1e48209ce6b8050fc3169d"}
Jul 28 19:53:42.269590 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:42.269459    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj" event={"ID":"e5025cdb-ab0d-457c-a948-11263ee3c692","Type":"ContainerStarted","Data":"a8ecfa886137797361f2946444e8e46189ab0125f409b5823ddfccaea5e375b1"}
Jul 28 19:53:42.269590 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:42.269522    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"
Jul 28 19:53:42.270800 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:42.270777    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h" event={"ID":"7dee60a2-ae6f-4126-acae-05b0ead764c0","Type":"ContainerStarted","Data":"d2672fac67030b125904287d73917cc778867276cfef1fa5fefabe85c1c2356f"}
Jul 28 19:53:42.270916 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:42.270818    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h" event={"ID":"7dee60a2-ae6f-4126-acae-05b0ead764c0","Type":"ContainerStarted","Data":"f13ba933142ca59e5c3d2a9d5bf7acb39eefdeb15756163631e9aa4e0e15d9a1"}
Jul 28 19:53:42.270916 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:42.270865    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"
Jul 28 19:53:42.285148 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:42.285109    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q" podStartSLOduration=1.285098306 podStartE2EDuration="1.285098306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:53:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:53:42.28351891 +0000 UTC m=+3070.230003798" watchObservedRunningTime="2026-07-28 19:53:42.285098306 +0000 UTC m=+3070.231583171"
Jul 28 19:53:42.299910 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:42.299873    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj" podStartSLOduration=1.299862553 podStartE2EDuration="1.299862553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:53:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:53:42.298829483 +0000 UTC m=+3070.245314366" watchObservedRunningTime="2026-07-28 19:53:42.299862553 +0000 UTC m=+3070.246347410"
Jul 28 19:53:42.315063 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:42.315030    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h" podStartSLOduration=1.8570467069999999 podStartE2EDuration="2.315017966s" totalImagesPullingTime="457.971259ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:53:42.313823239 +0000 UTC m=+3070.260308138" watchObservedRunningTime="2026-07-28 19:53:42.315017966 +0000 UTC m=+3070.261502834"
Jul 28 19:53:48.280254 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:48.280224    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"
Jul 28 19:53:53.276657 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:53.276625    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"
Jul 28 19:53:53.277119 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:53:53.276763    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"
Jul 28 19:54:10.914471 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:10.914440    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"]
Jul 28 19:54:10.918435 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:10.918416    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"
Jul 28 19:54:10.920811 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:10.920787    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-e29fb5-serving-cert\""
Jul 28 19:54:10.929814 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:10.929791    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"]
Jul 28 19:54:10.971205 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:10.971181    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn\" (UID: \"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"
Jul 28 19:54:10.971300 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:10.971252    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4n48\" (UniqueName: \"kubernetes.io/projected/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-kube-api-access-f4n48\") pod \"maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn\" (UID: \"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"
Jul 28 19:54:10.971343 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:10.971320    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn\" (UID: \"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"
Jul 28 19:54:11.072419 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.072391    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn\" (UID: \"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"
Jul 28 19:54:11.072548 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.072477    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4n48\" (UniqueName: \"kubernetes.io/projected/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-kube-api-access-f4n48\") pod \"maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn\" (UID: \"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"
Jul 28 19:54:11.072622 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.072560    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn\" (UID: \"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"
Jul 28 19:54:11.073189 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.073170    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn\" (UID: \"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"
Jul 28 19:54:11.074660 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.074638    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn\" (UID: \"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"
Jul 28 19:54:11.079561 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.079544    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4n48\" (UniqueName: \"kubernetes.io/projected/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-kube-api-access-f4n48\") pod \"maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn\" (UID: \"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"
Jul 28 19:54:11.232071 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.232049    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"
Jul 28 19:54:11.359444 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.359423    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"]
Jul 28 19:54:11.361656 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:54:11.361626    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9bce86b_34b1_4bbf_83cb_da0f33d0b0a0.slice/crio-724198b0f9223c21d48e45b23fffd5fed11743aaef6b263842811f593be870de WatchSource:0}: Error finding container 724198b0f9223c21d48e45b23fffd5fed11743aaef6b263842811f593be870de: Status 404 returned error can't find the container with id 724198b0f9223c21d48e45b23fffd5fed11743aaef6b263842811f593be870de
Jul 28 19:54:11.418971 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.418939    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn" event={"ID":"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0","Type":"ContainerStarted","Data":"724198b0f9223c21d48e45b23fffd5fed11743aaef6b263842811f593be870de"}
Jul 28 19:54:11.521240 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.521169    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"]
Jul 28 19:54:11.526143 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.526122    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"
Jul 28 19:54:11.528472 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.528453    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-e29fb5-dockercfg-qs47c\""
Jul 28 19:54:11.528472 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.528464    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-e29fb5\""
Jul 28 19:54:11.533000 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.532980    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"]
Jul 28 19:54:11.577273 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.577248    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a46bb93-f355-40b1-a7ca-1a4cbe34f974-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg\" (UID: \"6a46bb93-f355-40b1-a7ca-1a4cbe34f974\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"
Jul 28 19:54:11.577365 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.577289    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx87s\" (UniqueName: \"kubernetes.io/projected/6a46bb93-f355-40b1-a7ca-1a4cbe34f974-kube-api-access-vx87s\") pod \"payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg\" (UID: \"6a46bb93-f355-40b1-a7ca-1a4cbe34f974\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"
Jul 28 19:54:11.678568 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.678541    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a46bb93-f355-40b1-a7ca-1a4cbe34f974-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg\" (UID: \"6a46bb93-f355-40b1-a7ca-1a4cbe34f974\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"
Jul 28 19:54:11.678668 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.678578    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx87s\" (UniqueName: \"kubernetes.io/projected/6a46bb93-f355-40b1-a7ca-1a4cbe34f974-kube-api-access-vx87s\") pod \"payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg\" (UID: \"6a46bb93-f355-40b1-a7ca-1a4cbe34f974\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"
Jul 28 19:54:11.679193 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.679176    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a46bb93-f355-40b1-a7ca-1a4cbe34f974-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg\" (UID: \"6a46bb93-f355-40b1-a7ca-1a4cbe34f974\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"
Jul 28 19:54:11.685762 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.685741    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx87s\" (UniqueName: \"kubernetes.io/projected/6a46bb93-f355-40b1-a7ca-1a4cbe34f974-kube-api-access-vx87s\") pod \"payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg\" (UID: \"6a46bb93-f355-40b1-a7ca-1a4cbe34f974\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"
Jul 28 19:54:11.837648 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.837628    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"
Jul 28 19:54:11.961844 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:11.961818    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"]
Jul 28 19:54:11.963897 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:54:11.963873    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a46bb93_f355_40b1_a7ca_1a4cbe34f974.slice/crio-cce17b4fe07eb307491caefe67f3f67c602b46ff60bd4dfdb3a3ed88760fa3c0 WatchSource:0}: Error finding container cce17b4fe07eb307491caefe67f3f67c602b46ff60bd4dfdb3a3ed88760fa3c0: Status 404 returned error can't find the container with id cce17b4fe07eb307491caefe67f3f67c602b46ff60bd4dfdb3a3ed88760fa3c0
Jul 28 19:54:12.425851 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:12.425817    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg" event={"ID":"6a46bb93-f355-40b1-a7ca-1a4cbe34f974","Type":"ContainerStarted","Data":"a239264eb3271be26d1a3539a61850da87dbe1c480483505ee172f493aaf3c6a"}
Jul 28 19:54:12.426662 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:12.426083    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg" event={"ID":"6a46bb93-f355-40b1-a7ca-1a4cbe34f974","Type":"ContainerStarted","Data":"cce17b4fe07eb307491caefe67f3f67c602b46ff60bd4dfdb3a3ed88760fa3c0"}
Jul 28 19:54:12.426822 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:12.426737    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"
Jul 28 19:54:12.433555 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:12.433521    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn" event={"ID":"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0","Type":"ContainerStarted","Data":"755a9345bf5a4cfc537fd45308eb5002f96ef96e0ca502b81546cc2248d56bb1"}
Jul 28 19:54:12.433679 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:12.433651    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"
Jul 28 19:54:12.446263 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:12.446221    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg" podStartSLOduration=1.446211361 podStartE2EDuration="1.446211361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:54:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:54:12.44497379 +0000 UTC m=+3100.391458652" watchObservedRunningTime="2026-07-28 19:54:12.446211361 +0000 UTC m=+3100.392696227"
Jul 28 19:54:12.463277 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:12.462341    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn" podStartSLOduration=2.075574674 podStartE2EDuration="2.462325831s" totalImagesPullingTime="386.751157ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:54:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:54:12.461186327 +0000 UTC m=+3100.407671192" watchObservedRunningTime="2026-07-28 19:54:12.462325831 +0000 UTC m=+3100.408810699"
Jul 28 19:54:18.443684 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:18.443654    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"
Jul 28 19:54:23.440721 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:23.440685    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"
Jul 28 19:54:31.418598 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:54:31.418569    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-jdvzc_bec6473a-99eb-4911-9557-3ba753ed35eb/payload-processing/0.log"
Jul 28 19:55:28.612422 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:28.612345    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj_e5025cdb-ab0d-457c-a948-11263ee3c692/payload-processing/0.log"
Jul 28 19:55:28.731777 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:28.731753    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-jdvzc_bec6473a-99eb-4911-9557-3ba753ed35eb/payload-processing/0.log"
Jul 28 19:55:35.032769 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.032730    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"]
Jul 28 19:55:35.036640 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.036618    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"
Jul 28 19:55:35.039221 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.039196    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-ec8cba-dockercfg-9c7jl\""
Jul 28 19:55:35.039328 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.039207    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-ec8cba\""
Jul 28 19:55:35.044468 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.044412    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"]
Jul 28 19:55:35.171802 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.171781    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee31921-6bb1-4fe0-a13f-ee5f768f038b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47\" (UID: \"8ee31921-6bb1-4fe0-a13f-ee5f768f038b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"
Jul 28 19:55:35.171919 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.171815    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z9dn\" (UniqueName: \"kubernetes.io/projected/8ee31921-6bb1-4fe0-a13f-ee5f768f038b-kube-api-access-7z9dn\") pod \"payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47\" (UID: \"8ee31921-6bb1-4fe0-a13f-ee5f768f038b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"
Jul 28 19:55:35.272448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.272419    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee31921-6bb1-4fe0-a13f-ee5f768f038b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47\" (UID: \"8ee31921-6bb1-4fe0-a13f-ee5f768f038b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"
Jul 28 19:55:35.272591 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.272465    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z9dn\" (UniqueName: \"kubernetes.io/projected/8ee31921-6bb1-4fe0-a13f-ee5f768f038b-kube-api-access-7z9dn\") pod \"payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47\" (UID: \"8ee31921-6bb1-4fe0-a13f-ee5f768f038b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"
Jul 28 19:55:35.273278 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.273256    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee31921-6bb1-4fe0-a13f-ee5f768f038b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47\" (UID: \"8ee31921-6bb1-4fe0-a13f-ee5f768f038b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"
Jul 28 19:55:35.280252 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.280231    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z9dn\" (UniqueName: \"kubernetes.io/projected/8ee31921-6bb1-4fe0-a13f-ee5f768f038b-kube-api-access-7z9dn\") pod \"payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47\" (UID: \"8ee31921-6bb1-4fe0-a13f-ee5f768f038b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"
Jul 28 19:55:35.348974 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.348922    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"
Jul 28 19:55:35.474975 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.474946    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"]
Jul 28 19:55:35.477333 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:55:35.477302    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee31921_6bb1_4fe0_a13f_ee5f768f038b.slice/crio-57e5c90b9e21303a95f15c52bc4d2e1821d6ebc84c5acd083300f57432c226af WatchSource:0}: Error finding container 57e5c90b9e21303a95f15c52bc4d2e1821d6ebc84c5acd083300f57432c226af: Status 404 returned error can't find the container with id 57e5c90b9e21303a95f15c52bc4d2e1821d6ebc84c5acd083300f57432c226af
Jul 28 19:55:35.847990 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.847961    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47" event={"ID":"8ee31921-6bb1-4fe0-a13f-ee5f768f038b","Type":"ContainerStarted","Data":"89b7005807cbb5f5a6faab67533ed0fadd314b4906412cb70d660639442667b8"}
Jul 28 19:55:35.848135 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.848008    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47" event={"ID":"8ee31921-6bb1-4fe0-a13f-ee5f768f038b","Type":"ContainerStarted","Data":"57e5c90b9e21303a95f15c52bc4d2e1821d6ebc84c5acd083300f57432c226af"}
Jul 28 19:55:35.848135 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.848088    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"
Jul 28 19:55:35.863088 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:35.863045    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47" podStartSLOduration=0.863006825 podStartE2EDuration="863.006825ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:55:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:55:35.862423812 +0000 UTC m=+3183.808908677" watchObservedRunningTime="2026-07-28 19:55:35.863006825 +0000 UTC m=+3183.809491692"
Jul 28 19:55:46.854551 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:46.854516    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"
Jul 28 19:55:55.820575 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:55.820539    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"]
Jul 28 19:55:55.821003 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:55.820741    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47" podUID="8ee31921-6bb1-4fe0-a13f-ee5f768f038b" containerName="payload-processing" containerID="cri-o://89b7005807cbb5f5a6faab67533ed0fadd314b4906412cb70d660639442667b8" gracePeriod=30
Jul 28 19:55:55.956342 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:55.956312    2561 generic.go:356] "Generic (PLEG): container finished" podID="8ee31921-6bb1-4fe0-a13f-ee5f768f038b" containerID="89b7005807cbb5f5a6faab67533ed0fadd314b4906412cb70d660639442667b8" exitCode=0
Jul 28 19:55:55.956510 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:55.956400    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47" event={"ID":"8ee31921-6bb1-4fe0-a13f-ee5f768f038b","Type":"ContainerDied","Data":"89b7005807cbb5f5a6faab67533ed0fadd314b4906412cb70d660639442667b8"}
Jul 28 19:55:56.066520 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:56.066474    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"
Jul 28 19:55:56.248763 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:56.248739    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee31921-6bb1-4fe0-a13f-ee5f768f038b-plugins-config-volume\") pod \"8ee31921-6bb1-4fe0-a13f-ee5f768f038b\" (UID: \"8ee31921-6bb1-4fe0-a13f-ee5f768f038b\") "
Jul 28 19:55:56.248900 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:56.248773    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7z9dn\" (UniqueName: \"kubernetes.io/projected/8ee31921-6bb1-4fe0-a13f-ee5f768f038b-kube-api-access-7z9dn\") pod \"8ee31921-6bb1-4fe0-a13f-ee5f768f038b\" (UID: \"8ee31921-6bb1-4fe0-a13f-ee5f768f038b\") "
Jul 28 19:55:56.249074 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:56.249050    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ee31921-6bb1-4fe0-a13f-ee5f768f038b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8ee31921-6bb1-4fe0-a13f-ee5f768f038b" (UID: "8ee31921-6bb1-4fe0-a13f-ee5f768f038b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:55:56.250773 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:56.250753    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ee31921-6bb1-4fe0-a13f-ee5f768f038b-kube-api-access-7z9dn" (OuterVolumeSpecName: "kube-api-access-7z9dn") pod "8ee31921-6bb1-4fe0-a13f-ee5f768f038b" (UID: "8ee31921-6bb1-4fe0-a13f-ee5f768f038b"). InnerVolumeSpecName "kube-api-access-7z9dn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:55:56.349898 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:56.349874    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee31921-6bb1-4fe0-a13f-ee5f768f038b-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:55:56.349898 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:56.349896    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z9dn\" (UniqueName: \"kubernetes.io/projected/8ee31921-6bb1-4fe0-a13f-ee5f768f038b-kube-api-access-7z9dn\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:55:56.962883 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:56.962850    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47" event={"ID":"8ee31921-6bb1-4fe0-a13f-ee5f768f038b","Type":"ContainerDied","Data":"57e5c90b9e21303a95f15c52bc4d2e1821d6ebc84c5acd083300f57432c226af"}
Jul 28 19:55:56.963282 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:56.962919    2561 scope.go:117] "RemoveContainer" containerID="89b7005807cbb5f5a6faab67533ed0fadd314b4906412cb70d660639442667b8"
Jul 28 19:55:56.963282 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:56.962856    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"
Jul 28 19:55:56.978047 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:56.978021    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"]
Jul 28 19:55:56.980432 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:56.980408    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ec8cba-7956b775cb-g7m47"]
Jul 28 19:55:58.604229 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:55:58.604179    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ee31921-6bb1-4fe0-a13f-ee5f768f038b" path="/var/lib/kubelet/pods/8ee31921-6bb1-4fe0-a13f-ee5f768f038b/volumes"
Jul 28 19:56:23.403400 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.403366    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"]
Jul 28 19:56:23.403914 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.403575    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h" podUID="7dee60a2-ae6f-4126-acae-05b0ead764c0" containerName="maas-api" containerID="cri-o://d2672fac67030b125904287d73917cc778867276cfef1fa5fefabe85c1c2356f" gracePeriod=30
Jul 28 19:56:23.456158 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.456131    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"]
Jul 28 19:56:23.456381 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.456340    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj" podUID="e5025cdb-ab0d-457c-a948-11263ee3c692" containerName="payload-processing" containerID="cri-o://565aba29fe1f4ee9d01656c33a36d7f9b654911f6d1e48209ce6b8050fc3169d" gracePeriod=30
Jul 28 19:56:23.463322 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.463297    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"]
Jul 28 19:56:23.463536 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.463506    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q" podUID="93579aec-c805-4eda-afdd-77830b1f5fa0" containerName="payload-pre-processing" containerID="cri-o://b92f225b91511297af3dc0526a4c058db59085ef3d409e6128904173ca7859ed" gracePeriod=30
Jul 28 19:56:23.726067 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.726047    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"
Jul 28 19:56:23.739897 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.739875    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"
Jul 28 19:56:23.751378 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.751360    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"
Jul 28 19:56:23.770993 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.770609    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dee60a2-ae6f-4126-acae-05b0ead764c0-openshift-service-ca\") pod \"7dee60a2-ae6f-4126-acae-05b0ead764c0\" (UID: \"7dee60a2-ae6f-4126-acae-05b0ead764c0\") "
Jul 28 19:56:23.770993 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.770644    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dee60a2-ae6f-4126-acae-05b0ead764c0-maas-api-tls\") pod \"7dee60a2-ae6f-4126-acae-05b0ead764c0\" (UID: \"7dee60a2-ae6f-4126-acae-05b0ead764c0\") "
Jul 28 19:56:23.770993 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.770716    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4t9d\" (UniqueName: \"kubernetes.io/projected/7dee60a2-ae6f-4126-acae-05b0ead764c0-kube-api-access-m4t9d\") pod \"7dee60a2-ae6f-4126-acae-05b0ead764c0\" (UID: \"7dee60a2-ae6f-4126-acae-05b0ead764c0\") "
Jul 28 19:56:23.771263 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.771029    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dee60a2-ae6f-4126-acae-05b0ead764c0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7dee60a2-ae6f-4126-acae-05b0ead764c0" (UID: "7dee60a2-ae6f-4126-acae-05b0ead764c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:56:23.771669 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.771443    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5025cdb-ab0d-457c-a948-11263ee3c692-plugins-config-volume\") pod \"e5025cdb-ab0d-457c-a948-11263ee3c692\" (UID: \"e5025cdb-ab0d-457c-a948-11263ee3c692\") "
Jul 28 19:56:23.771669 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.771510    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldl9t\" (UniqueName: \"kubernetes.io/projected/e5025cdb-ab0d-457c-a948-11263ee3c692-kube-api-access-ldl9t\") pod \"e5025cdb-ab0d-457c-a948-11263ee3c692\" (UID: \"e5025cdb-ab0d-457c-a948-11263ee3c692\") "
Jul 28 19:56:23.773018 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.772179    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dee60a2-ae6f-4126-acae-05b0ead764c0-openshift-service-ca\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:56:23.773018 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.772933    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5025cdb-ab0d-457c-a948-11263ee3c692-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e5025cdb-ab0d-457c-a948-11263ee3c692" (UID: "e5025cdb-ab0d-457c-a948-11263ee3c692"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:56:23.773703 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.773680    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7dee60a2-ae6f-4126-acae-05b0ead764c0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7dee60a2-ae6f-4126-acae-05b0ead764c0" (UID: "7dee60a2-ae6f-4126-acae-05b0ead764c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:56:23.775512 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.774794    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dee60a2-ae6f-4126-acae-05b0ead764c0-kube-api-access-m4t9d" (OuterVolumeSpecName: "kube-api-access-m4t9d") pod "7dee60a2-ae6f-4126-acae-05b0ead764c0" (UID: "7dee60a2-ae6f-4126-acae-05b0ead764c0"). InnerVolumeSpecName "kube-api-access-m4t9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:56:23.775780 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.775760    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5025cdb-ab0d-457c-a948-11263ee3c692-kube-api-access-ldl9t" (OuterVolumeSpecName: "kube-api-access-ldl9t") pod "e5025cdb-ab0d-457c-a948-11263ee3c692" (UID: "e5025cdb-ab0d-457c-a948-11263ee3c692"). InnerVolumeSpecName "kube-api-access-ldl9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:56:23.873003 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.872980    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xclm\" (UniqueName: \"kubernetes.io/projected/93579aec-c805-4eda-afdd-77830b1f5fa0-kube-api-access-8xclm\") pod \"93579aec-c805-4eda-afdd-77830b1f5fa0\" (UID: \"93579aec-c805-4eda-afdd-77830b1f5fa0\") "
Jul 28 19:56:23.873098 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.873086    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93579aec-c805-4eda-afdd-77830b1f5fa0-plugins-config-volume\") pod \"93579aec-c805-4eda-afdd-77830b1f5fa0\" (UID: \"93579aec-c805-4eda-afdd-77830b1f5fa0\") "
Jul 28 19:56:23.873306 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.873283    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dee60a2-ae6f-4126-acae-05b0ead764c0-maas-api-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:56:23.873363 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.873307    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4t9d\" (UniqueName: \"kubernetes.io/projected/7dee60a2-ae6f-4126-acae-05b0ead764c0-kube-api-access-m4t9d\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:56:23.873363 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.873322    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5025cdb-ab0d-457c-a948-11263ee3c692-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:56:23.873363 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.873336    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldl9t\" (UniqueName: \"kubernetes.io/projected/e5025cdb-ab0d-457c-a948-11263ee3c692-kube-api-access-ldl9t\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:56:23.873457 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.873417    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93579aec-c805-4eda-afdd-77830b1f5fa0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "93579aec-c805-4eda-afdd-77830b1f5fa0" (UID: "93579aec-c805-4eda-afdd-77830b1f5fa0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:56:23.874702 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.874685    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93579aec-c805-4eda-afdd-77830b1f5fa0-kube-api-access-8xclm" (OuterVolumeSpecName: "kube-api-access-8xclm") pod "93579aec-c805-4eda-afdd-77830b1f5fa0" (UID: "93579aec-c805-4eda-afdd-77830b1f5fa0"). InnerVolumeSpecName "kube-api-access-8xclm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:56:23.974214 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.974192    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93579aec-c805-4eda-afdd-77830b1f5fa0-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:56:23.974214 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:23.974215    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xclm\" (UniqueName: \"kubernetes.io/projected/93579aec-c805-4eda-afdd-77830b1f5fa0-kube-api-access-8xclm\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:56:24.096281 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.096250    2561 generic.go:356] "Generic (PLEG): container finished" podID="93579aec-c805-4eda-afdd-77830b1f5fa0" containerID="b92f225b91511297af3dc0526a4c058db59085ef3d409e6128904173ca7859ed" exitCode=0
Jul 28 19:56:24.096400 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.096288    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q" event={"ID":"93579aec-c805-4eda-afdd-77830b1f5fa0","Type":"ContainerDied","Data":"b92f225b91511297af3dc0526a4c058db59085ef3d409e6128904173ca7859ed"}
Jul 28 19:56:24.096400 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.096311    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"
Jul 28 19:56:24.096400 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.096366    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q" event={"ID":"93579aec-c805-4eda-afdd-77830b1f5fa0","Type":"ContainerDied","Data":"e218f023006ace48efb58aba38515fa0962d475ab7bd97287a146013c932db36"}
Jul 28 19:56:24.096400 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.096399    2561 scope.go:117] "RemoveContainer" containerID="b92f225b91511297af3dc0526a4c058db59085ef3d409e6128904173ca7859ed"
Jul 28 19:56:24.098116 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.098087    2561 generic.go:356] "Generic (PLEG): container finished" podID="e5025cdb-ab0d-457c-a948-11263ee3c692" containerID="565aba29fe1f4ee9d01656c33a36d7f9b654911f6d1e48209ce6b8050fc3169d" exitCode=0
Jul 28 19:56:24.098236 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.098135    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"
Jul 28 19:56:24.098236 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.098173    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj" event={"ID":"e5025cdb-ab0d-457c-a948-11263ee3c692","Type":"ContainerDied","Data":"565aba29fe1f4ee9d01656c33a36d7f9b654911f6d1e48209ce6b8050fc3169d"}
Jul 28 19:56:24.098236 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.098228    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj" event={"ID":"e5025cdb-ab0d-457c-a948-11263ee3c692","Type":"ContainerDied","Data":"a8ecfa886137797361f2946444e8e46189ab0125f409b5823ddfccaea5e375b1"}
Jul 28 19:56:24.103036 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.102105    2561 generic.go:356] "Generic (PLEG): container finished" podID="7dee60a2-ae6f-4126-acae-05b0ead764c0" containerID="d2672fac67030b125904287d73917cc778867276cfef1fa5fefabe85c1c2356f" exitCode=0
Jul 28 19:56:24.103036 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.102615    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h" event={"ID":"7dee60a2-ae6f-4126-acae-05b0ead764c0","Type":"ContainerDied","Data":"d2672fac67030b125904287d73917cc778867276cfef1fa5fefabe85c1c2356f"}
Jul 28 19:56:24.103036 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.102721    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h" event={"ID":"7dee60a2-ae6f-4126-acae-05b0ead764c0","Type":"ContainerDied","Data":"f13ba933142ca59e5c3d2a9d5bf7acb39eefdeb15756163631e9aa4e0e15d9a1"}
Jul 28 19:56:24.103036 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.102924    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"
Jul 28 19:56:24.115987 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.115964    2561 scope.go:117] "RemoveContainer" containerID="b92f225b91511297af3dc0526a4c058db59085ef3d409e6128904173ca7859ed"
Jul 28 19:56:24.116282 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:56:24.116253    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b92f225b91511297af3dc0526a4c058db59085ef3d409e6128904173ca7859ed\": container with ID starting with b92f225b91511297af3dc0526a4c058db59085ef3d409e6128904173ca7859ed not found: ID does not exist" containerID="b92f225b91511297af3dc0526a4c058db59085ef3d409e6128904173ca7859ed"
Jul 28 19:56:24.116371 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.116295    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b92f225b91511297af3dc0526a4c058db59085ef3d409e6128904173ca7859ed"} err="failed to get container status \"b92f225b91511297af3dc0526a4c058db59085ef3d409e6128904173ca7859ed\": rpc error: code = NotFound desc = could not find container \"b92f225b91511297af3dc0526a4c058db59085ef3d409e6128904173ca7859ed\": container with ID starting with b92f225b91511297af3dc0526a4c058db59085ef3d409e6128904173ca7859ed not found: ID does not exist"
Jul 28 19:56:24.116371 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.116326    2561 scope.go:117] "RemoveContainer" containerID="565aba29fe1f4ee9d01656c33a36d7f9b654911f6d1e48209ce6b8050fc3169d"
Jul 28 19:56:24.124516 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.124495    2561 scope.go:117] "RemoveContainer" containerID="565aba29fe1f4ee9d01656c33a36d7f9b654911f6d1e48209ce6b8050fc3169d"
Jul 28 19:56:24.124767 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:56:24.124751    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"565aba29fe1f4ee9d01656c33a36d7f9b654911f6d1e48209ce6b8050fc3169d\": container with ID starting with 565aba29fe1f4ee9d01656c33a36d7f9b654911f6d1e48209ce6b8050fc3169d not found: ID does not exist" containerID="565aba29fe1f4ee9d01656c33a36d7f9b654911f6d1e48209ce6b8050fc3169d"
Jul 28 19:56:24.124832 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.124772    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"565aba29fe1f4ee9d01656c33a36d7f9b654911f6d1e48209ce6b8050fc3169d"} err="failed to get container status \"565aba29fe1f4ee9d01656c33a36d7f9b654911f6d1e48209ce6b8050fc3169d\": rpc error: code = NotFound desc = could not find container \"565aba29fe1f4ee9d01656c33a36d7f9b654911f6d1e48209ce6b8050fc3169d\": container with ID starting with 565aba29fe1f4ee9d01656c33a36d7f9b654911f6d1e48209ce6b8050fc3169d not found: ID does not exist"
Jul 28 19:56:24.124832 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.124787    2561 scope.go:117] "RemoveContainer" containerID="d2672fac67030b125904287d73917cc778867276cfef1fa5fefabe85c1c2356f"
Jul 28 19:56:24.126515 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.126477    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"]
Jul 28 19:56:24.132590 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.132534    2561 scope.go:117] "RemoveContainer" containerID="d2672fac67030b125904287d73917cc778867276cfef1fa5fefabe85c1c2356f"
Jul 28 19:56:24.132821 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:56:24.132802    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2672fac67030b125904287d73917cc778867276cfef1fa5fefabe85c1c2356f\": container with ID starting with d2672fac67030b125904287d73917cc778867276cfef1fa5fefabe85c1c2356f not found: ID does not exist" containerID="d2672fac67030b125904287d73917cc778867276cfef1fa5fefabe85c1c2356f"
Jul 28 19:56:24.132885 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.132833    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2672fac67030b125904287d73917cc778867276cfef1fa5fefabe85c1c2356f"} err="failed to get container status \"d2672fac67030b125904287d73917cc778867276cfef1fa5fefabe85c1c2356f\": rpc error: code = NotFound desc = could not find container \"d2672fac67030b125904287d73917cc778867276cfef1fa5fefabe85c1c2356f\": container with ID starting with d2672fac67030b125904287d73917cc778867276cfef1fa5fefabe85c1c2356f not found: ID does not exist"
Jul 28 19:56:24.133146 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.133126    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cf1393-58674f6784-q2v7q"]
Jul 28 19:56:24.142581 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.142561    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"]
Jul 28 19:56:24.146542 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.146522    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cf1393-64d66d6cd4-76xjj"]
Jul 28 19:56:24.156877 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.156849    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"]
Jul 28 19:56:24.160861 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.160841    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-cf1393-646f5cf8c6-zlh8h"]
Jul 28 19:56:24.604347 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.604323    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dee60a2-ae6f-4126-acae-05b0ead764c0" path="/var/lib/kubelet/pods/7dee60a2-ae6f-4126-acae-05b0ead764c0/volumes"
Jul 28 19:56:24.604740 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.604725    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93579aec-c805-4eda-afdd-77830b1f5fa0" path="/var/lib/kubelet/pods/93579aec-c805-4eda-afdd-77830b1f5fa0/volumes"
Jul 28 19:56:24.605081 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:56:24.605068    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5025cdb-ab0d-457c-a948-11263ee3c692" path="/var/lib/kubelet/pods/e5025cdb-ab0d-457c-a948-11263ee3c692/volumes"
Jul 28 19:57:19.096716 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.096679    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"]
Jul 28 19:57:19.097330 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.096940    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn" podUID="e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0" containerName="maas-api" containerID="cri-o://755a9345bf5a4cfc537fd45308eb5002f96ef96e0ca502b81546cc2248d56bb1" gracePeriod=30
Jul 28 19:57:19.163774 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.163748    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"]
Jul 28 19:57:19.163943 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.163920    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg" podUID="6a46bb93-f355-40b1-a7ca-1a4cbe34f974" containerName="payload-pre-processing" containerID="cri-o://a239264eb3271be26d1a3539a61850da87dbe1c480483505ee172f493aaf3c6a" gracePeriod=30
Jul 28 19:57:19.389679 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.389495    2561 generic.go:356] "Generic (PLEG): container finished" podID="6a46bb93-f355-40b1-a7ca-1a4cbe34f974" containerID="a239264eb3271be26d1a3539a61850da87dbe1c480483505ee172f493aaf3c6a" exitCode=0
Jul 28 19:57:19.389679 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.389600    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg" event={"ID":"6a46bb93-f355-40b1-a7ca-1a4cbe34f974","Type":"ContainerDied","Data":"a239264eb3271be26d1a3539a61850da87dbe1c480483505ee172f493aaf3c6a"}
Jul 28 19:57:19.391803 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.391693    2561 generic.go:356] "Generic (PLEG): container finished" podID="e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0" containerID="755a9345bf5a4cfc537fd45308eb5002f96ef96e0ca502b81546cc2248d56bb1" exitCode=0
Jul 28 19:57:19.391803 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.391752    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn" event={"ID":"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0","Type":"ContainerDied","Data":"755a9345bf5a4cfc537fd45308eb5002f96ef96e0ca502b81546cc2248d56bb1"}
Jul 28 19:57:19.421450 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.421432    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"
Jul 28 19:57:19.445295 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.445277    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"
Jul 28 19:57:19.512190 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.512167    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4n48\" (UniqueName: \"kubernetes.io/projected/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-kube-api-access-f4n48\") pod \"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0\" (UID: \"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0\") "
Jul 28 19:57:19.512290 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.512203    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a46bb93-f355-40b1-a7ca-1a4cbe34f974-plugins-config-volume\") pod \"6a46bb93-f355-40b1-a7ca-1a4cbe34f974\" (UID: \"6a46bb93-f355-40b1-a7ca-1a4cbe34f974\") "
Jul 28 19:57:19.512290 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.512228    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-maas-api-tls\") pod \"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0\" (UID: \"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0\") "
Jul 28 19:57:19.512290 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.512250    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-openshift-service-ca\") pod \"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0\" (UID: \"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0\") "
Jul 28 19:57:19.512455 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.512330    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vx87s\" (UniqueName: \"kubernetes.io/projected/6a46bb93-f355-40b1-a7ca-1a4cbe34f974-kube-api-access-vx87s\") pod \"6a46bb93-f355-40b1-a7ca-1a4cbe34f974\" (UID: \"6a46bb93-f355-40b1-a7ca-1a4cbe34f974\") "
Jul 28 19:57:19.512640 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.512621    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a46bb93-f355-40b1-a7ca-1a4cbe34f974-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6a46bb93-f355-40b1-a7ca-1a4cbe34f974" (UID: "6a46bb93-f355-40b1-a7ca-1a4cbe34f974"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:57:19.512711 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.512651    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0" (UID: "e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:57:19.514309 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.514287    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0" (UID: "e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:57:19.514398 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.514337    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-kube-api-access-f4n48" (OuterVolumeSpecName: "kube-api-access-f4n48") pod "e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0" (UID: "e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0"). InnerVolumeSpecName "kube-api-access-f4n48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:19.514398 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.514359    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a46bb93-f355-40b1-a7ca-1a4cbe34f974-kube-api-access-vx87s" (OuterVolumeSpecName: "kube-api-access-vx87s") pod "6a46bb93-f355-40b1-a7ca-1a4cbe34f974" (UID: "6a46bb93-f355-40b1-a7ca-1a4cbe34f974"). InnerVolumeSpecName "kube-api-access-vx87s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:19.612963 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.612905    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4n48\" (UniqueName: \"kubernetes.io/projected/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-kube-api-access-f4n48\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:19.612963 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.612928    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a46bb93-f355-40b1-a7ca-1a4cbe34f974-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:19.612963 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.612937    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-maas-api-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:19.612963 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.612947    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0-openshift-service-ca\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:19.612963 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:19.612957    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx87s\" (UniqueName: \"kubernetes.io/projected/6a46bb93-f355-40b1-a7ca-1a4cbe34f974-kube-api-access-vx87s\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:20.403450 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:20.403380    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"
Jul 28 19:57:20.403450 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:20.403401    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg" event={"ID":"6a46bb93-f355-40b1-a7ca-1a4cbe34f974","Type":"ContainerDied","Data":"cce17b4fe07eb307491caefe67f3f67c602b46ff60bd4dfdb3a3ed88760fa3c0"}
Jul 28 19:57:20.403911 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:20.403504    2561 scope.go:117] "RemoveContainer" containerID="a239264eb3271be26d1a3539a61850da87dbe1c480483505ee172f493aaf3c6a"
Jul 28 19:57:20.404795 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:20.404775    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn" event={"ID":"e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0","Type":"ContainerDied","Data":"724198b0f9223c21d48e45b23fffd5fed11743aaef6b263842811f593be870de"}
Jul 28 19:57:20.404795 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:20.404788    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"
Jul 28 19:57:20.412954 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:20.412930    2561 scope.go:117] "RemoveContainer" containerID="755a9345bf5a4cfc537fd45308eb5002f96ef96e0ca502b81546cc2248d56bb1"
Jul 28 19:57:20.426705 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:20.426679    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"]
Jul 28 19:57:20.430600 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:20.430581    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e29fb5-5fb896d84-b8cmg"]
Jul 28 19:57:20.440804 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:20.440786    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"]
Jul 28 19:57:20.444269 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:20.444251    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e29fb5-59458f7995-mgqbn"]
Jul 28 19:57:20.604297 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:20.604274    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a46bb93-f355-40b1-a7ca-1a4cbe34f974" path="/var/lib/kubelet/pods/6a46bb93-f355-40b1-a7ca-1a4cbe34f974/volumes"
Jul 28 19:57:20.604689 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:20.604673    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0" path="/var/lib/kubelet/pods/e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0/volumes"
Jul 28 19:57:31.881827 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:31.881796    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"]
Jul 28 19:57:31.882263 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:31.882014    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h" podUID="381624bd-7373-48b0-9fb0-9b6b8d792790" containerName="maas-api" containerID="cri-o://985f192328a5f114a22b0b5a54d7bff3c07f5920ab25e1eeb1b7d13e0579be45" gracePeriod=30
Jul 28 19:57:32.027632 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.027603    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"]
Jul 28 19:57:32.027852 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.027817    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9" podUID="ddb26822-8d56-4554-b097-efbc3e9a99bd" containerName="payload-processing" containerID="cri-o://629df25e33cbb346c9f664e1ae5a45d9464b445d419c33e621931b5f4bc5250c" gracePeriod=30
Jul 28 19:57:32.033753 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.033307    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"]
Jul 28 19:57:32.033753 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.033526    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98" podUID="5911e907-05c1-45d8-aefc-984c393ff259" containerName="payload-pre-processing" containerID="cri-o://7b62c27578c5d87d51e52a34cf7cc6ec3ba74e20804c50d59f416a785d06ffbb" gracePeriod=30
Jul 28 19:57:32.170475 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.170447    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"
Jul 28 19:57:32.298300 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.298278    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"
Jul 28 19:57:32.300790 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.300771    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"
Jul 28 19:57:32.317674 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.317645    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7f65\" (UniqueName: \"kubernetes.io/projected/381624bd-7373-48b0-9fb0-9b6b8d792790-kube-api-access-m7f65\") pod \"381624bd-7373-48b0-9fb0-9b6b8d792790\" (UID: \"381624bd-7373-48b0-9fb0-9b6b8d792790\") "
Jul 28 19:57:32.317788 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.317714    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/381624bd-7373-48b0-9fb0-9b6b8d792790-openshift-service-ca\") pod \"381624bd-7373-48b0-9fb0-9b6b8d792790\" (UID: \"381624bd-7373-48b0-9fb0-9b6b8d792790\") "
Jul 28 19:57:32.317860 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.317812    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/381624bd-7373-48b0-9fb0-9b6b8d792790-maas-api-tls\") pod \"381624bd-7373-48b0-9fb0-9b6b8d792790\" (UID: \"381624bd-7373-48b0-9fb0-9b6b8d792790\") "
Jul 28 19:57:32.318110 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.318068    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/381624bd-7373-48b0-9fb0-9b6b8d792790-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "381624bd-7373-48b0-9fb0-9b6b8d792790" (UID: "381624bd-7373-48b0-9fb0-9b6b8d792790"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:57:32.319796 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.319776    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/381624bd-7373-48b0-9fb0-9b6b8d792790-kube-api-access-m7f65" (OuterVolumeSpecName: "kube-api-access-m7f65") pod "381624bd-7373-48b0-9fb0-9b6b8d792790" (UID: "381624bd-7373-48b0-9fb0-9b6b8d792790"). InnerVolumeSpecName "kube-api-access-m7f65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:32.319966 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.319943    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/381624bd-7373-48b0-9fb0-9b6b8d792790-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "381624bd-7373-48b0-9fb0-9b6b8d792790" (UID: "381624bd-7373-48b0-9fb0-9b6b8d792790"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:57:32.419141 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.419085    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5911e907-05c1-45d8-aefc-984c393ff259-plugins-config-volume\") pod \"5911e907-05c1-45d8-aefc-984c393ff259\" (UID: \"5911e907-05c1-45d8-aefc-984c393ff259\") "
Jul 28 19:57:32.419141 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.419117    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddb26822-8d56-4554-b097-efbc3e9a99bd-plugins-config-volume\") pod \"ddb26822-8d56-4554-b097-efbc3e9a99bd\" (UID: \"ddb26822-8d56-4554-b097-efbc3e9a99bd\") "
Jul 28 19:57:32.419325 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.419174    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpj7p\" (UniqueName: \"kubernetes.io/projected/ddb26822-8d56-4554-b097-efbc3e9a99bd-kube-api-access-hpj7p\") pod \"ddb26822-8d56-4554-b097-efbc3e9a99bd\" (UID: \"ddb26822-8d56-4554-b097-efbc3e9a99bd\") "
Jul 28 19:57:32.419325 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.419193    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9597\" (UniqueName: \"kubernetes.io/projected/5911e907-05c1-45d8-aefc-984c393ff259-kube-api-access-j9597\") pod \"5911e907-05c1-45d8-aefc-984c393ff259\" (UID: \"5911e907-05c1-45d8-aefc-984c393ff259\") "
Jul 28 19:57:32.419450 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.419432    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7f65\" (UniqueName: \"kubernetes.io/projected/381624bd-7373-48b0-9fb0-9b6b8d792790-kube-api-access-m7f65\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:32.419531 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.419456    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/381624bd-7373-48b0-9fb0-9b6b8d792790-openshift-service-ca\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:32.419531 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.419468    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/381624bd-7373-48b0-9fb0-9b6b8d792790-maas-api-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:32.419531 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.419502    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5911e907-05c1-45d8-aefc-984c393ff259-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5911e907-05c1-45d8-aefc-984c393ff259" (UID: "5911e907-05c1-45d8-aefc-984c393ff259"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:57:32.419629 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.419548    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddb26822-8d56-4554-b097-efbc3e9a99bd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ddb26822-8d56-4554-b097-efbc3e9a99bd" (UID: "ddb26822-8d56-4554-b097-efbc3e9a99bd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:57:32.421106 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.421089    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5911e907-05c1-45d8-aefc-984c393ff259-kube-api-access-j9597" (OuterVolumeSpecName: "kube-api-access-j9597") pod "5911e907-05c1-45d8-aefc-984c393ff259" (UID: "5911e907-05c1-45d8-aefc-984c393ff259"). InnerVolumeSpecName "kube-api-access-j9597". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:32.421196 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.421120    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddb26822-8d56-4554-b097-efbc3e9a99bd-kube-api-access-hpj7p" (OuterVolumeSpecName: "kube-api-access-hpj7p") pod "ddb26822-8d56-4554-b097-efbc3e9a99bd" (UID: "ddb26822-8d56-4554-b097-efbc3e9a99bd"). InnerVolumeSpecName "kube-api-access-hpj7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:32.465172 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.465149    2561 generic.go:356] "Generic (PLEG): container finished" podID="381624bd-7373-48b0-9fb0-9b6b8d792790" containerID="985f192328a5f114a22b0b5a54d7bff3c07f5920ab25e1eeb1b7d13e0579be45" exitCode=0
Jul 28 19:57:32.465253 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.465191    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"
Jul 28 19:57:32.465253 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.465236    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h" event={"ID":"381624bd-7373-48b0-9fb0-9b6b8d792790","Type":"ContainerDied","Data":"985f192328a5f114a22b0b5a54d7bff3c07f5920ab25e1eeb1b7d13e0579be45"}
Jul 28 19:57:32.465341 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.465308    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h" event={"ID":"381624bd-7373-48b0-9fb0-9b6b8d792790","Type":"ContainerDied","Data":"354a2549ee5db229a3e684188de48696ee6413515d0b40c18b25907c4500a299"}
Jul 28 19:57:32.465341 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.465332    2561 scope.go:117] "RemoveContainer" containerID="985f192328a5f114a22b0b5a54d7bff3c07f5920ab25e1eeb1b7d13e0579be45"
Jul 28 19:57:32.466475 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.466452    2561 generic.go:356] "Generic (PLEG): container finished" podID="ddb26822-8d56-4554-b097-efbc3e9a99bd" containerID="629df25e33cbb346c9f664e1ae5a45d9464b445d419c33e621931b5f4bc5250c" exitCode=0
Jul 28 19:57:32.466584 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.466513    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"
Jul 28 19:57:32.466638 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.466510    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9" event={"ID":"ddb26822-8d56-4554-b097-efbc3e9a99bd","Type":"ContainerDied","Data":"629df25e33cbb346c9f664e1ae5a45d9464b445d419c33e621931b5f4bc5250c"}
Jul 28 19:57:32.466701 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.466676    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9" event={"ID":"ddb26822-8d56-4554-b097-efbc3e9a99bd","Type":"ContainerDied","Data":"45366f8fc4c537f655924ca0bb184527d5c61cd03256bd287a2e44f22c9328a0"}
Jul 28 19:57:32.467899 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.467876    2561 generic.go:356] "Generic (PLEG): container finished" podID="5911e907-05c1-45d8-aefc-984c393ff259" containerID="7b62c27578c5d87d51e52a34cf7cc6ec3ba74e20804c50d59f416a785d06ffbb" exitCode=0
Jul 28 19:57:32.467986 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.467920    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"
Jul 28 19:57:32.467986 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.467965    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98" event={"ID":"5911e907-05c1-45d8-aefc-984c393ff259","Type":"ContainerDied","Data":"7b62c27578c5d87d51e52a34cf7cc6ec3ba74e20804c50d59f416a785d06ffbb"}
Jul 28 19:57:32.468060 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.468019    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98" event={"ID":"5911e907-05c1-45d8-aefc-984c393ff259","Type":"ContainerDied","Data":"c3502eac87100e020ce79b52f918daa100fc7efe0f1ab160f6571f52991d8164"}
Jul 28 19:57:32.473933 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.473913    2561 scope.go:117] "RemoveContainer" containerID="985f192328a5f114a22b0b5a54d7bff3c07f5920ab25e1eeb1b7d13e0579be45"
Jul 28 19:57:32.474179 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:57:32.474161    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"985f192328a5f114a22b0b5a54d7bff3c07f5920ab25e1eeb1b7d13e0579be45\": container with ID starting with 985f192328a5f114a22b0b5a54d7bff3c07f5920ab25e1eeb1b7d13e0579be45 not found: ID does not exist" containerID="985f192328a5f114a22b0b5a54d7bff3c07f5920ab25e1eeb1b7d13e0579be45"
Jul 28 19:57:32.474255 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.474185    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"985f192328a5f114a22b0b5a54d7bff3c07f5920ab25e1eeb1b7d13e0579be45"} err="failed to get container status \"985f192328a5f114a22b0b5a54d7bff3c07f5920ab25e1eeb1b7d13e0579be45\": rpc error: code = NotFound desc = could not find container \"985f192328a5f114a22b0b5a54d7bff3c07f5920ab25e1eeb1b7d13e0579be45\": container with ID starting with 985f192328a5f114a22b0b5a54d7bff3c07f5920ab25e1eeb1b7d13e0579be45 not found: ID does not exist"
Jul 28 19:57:32.474255 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.474201    2561 scope.go:117] "RemoveContainer" containerID="629df25e33cbb346c9f664e1ae5a45d9464b445d419c33e621931b5f4bc5250c"
Jul 28 19:57:32.482319 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.482304    2561 scope.go:117] "RemoveContainer" containerID="629df25e33cbb346c9f664e1ae5a45d9464b445d419c33e621931b5f4bc5250c"
Jul 28 19:57:32.482566 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:57:32.482545    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"629df25e33cbb346c9f664e1ae5a45d9464b445d419c33e621931b5f4bc5250c\": container with ID starting with 629df25e33cbb346c9f664e1ae5a45d9464b445d419c33e621931b5f4bc5250c not found: ID does not exist" containerID="629df25e33cbb346c9f664e1ae5a45d9464b445d419c33e621931b5f4bc5250c"
Jul 28 19:57:32.482644 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.482571    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"629df25e33cbb346c9f664e1ae5a45d9464b445d419c33e621931b5f4bc5250c"} err="failed to get container status \"629df25e33cbb346c9f664e1ae5a45d9464b445d419c33e621931b5f4bc5250c\": rpc error: code = NotFound desc = could not find container \"629df25e33cbb346c9f664e1ae5a45d9464b445d419c33e621931b5f4bc5250c\": container with ID starting with 629df25e33cbb346c9f664e1ae5a45d9464b445d419c33e621931b5f4bc5250c not found: ID does not exist"
Jul 28 19:57:32.482644 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.482585    2561 scope.go:117] "RemoveContainer" containerID="7b62c27578c5d87d51e52a34cf7cc6ec3ba74e20804c50d59f416a785d06ffbb"
Jul 28 19:57:32.487302 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.487277    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"]
Jul 28 19:57:32.489740 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.489724    2561 scope.go:117] "RemoveContainer" containerID="7b62c27578c5d87d51e52a34cf7cc6ec3ba74e20804c50d59f416a785d06ffbb"
Jul 28 19:57:32.489841 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.489819    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ec81e7-56b759d9d-72p9h"]
Jul 28 19:57:32.489975 ip-10-0-137-70 kubenswrapper[2561]: E0728 19:57:32.489958    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b62c27578c5d87d51e52a34cf7cc6ec3ba74e20804c50d59f416a785d06ffbb\": container with ID starting with 7b62c27578c5d87d51e52a34cf7cc6ec3ba74e20804c50d59f416a785d06ffbb not found: ID does not exist" containerID="7b62c27578c5d87d51e52a34cf7cc6ec3ba74e20804c50d59f416a785d06ffbb"
Jul 28 19:57:32.490022 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.489981    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b62c27578c5d87d51e52a34cf7cc6ec3ba74e20804c50d59f416a785d06ffbb"} err="failed to get container status \"7b62c27578c5d87d51e52a34cf7cc6ec3ba74e20804c50d59f416a785d06ffbb\": rpc error: code = NotFound desc = could not find container \"7b62c27578c5d87d51e52a34cf7cc6ec3ba74e20804c50d59f416a785d06ffbb\": container with ID starting with 7b62c27578c5d87d51e52a34cf7cc6ec3ba74e20804c50d59f416a785d06ffbb not found: ID does not exist"
Jul 28 19:57:32.499530 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.499510    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"]
Jul 28 19:57:32.503833 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.503816    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-ec81e7-5b79958d44-kqd98"]
Jul 28 19:57:32.512757 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.512740    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"]
Jul 28 19:57:32.516688 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.516670    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ec81e7-857cd8c598-98wd9"]
Jul 28 19:57:32.520524 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.520506    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5911e907-05c1-45d8-aefc-984c393ff259-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:32.520589 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.520525    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddb26822-8d56-4554-b097-efbc3e9a99bd-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:32.520589 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.520535    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpj7p\" (UniqueName: \"kubernetes.io/projected/ddb26822-8d56-4554-b097-efbc3e9a99bd-kube-api-access-hpj7p\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:32.520589 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.520544    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9597\" (UniqueName: \"kubernetes.io/projected/5911e907-05c1-45d8-aefc-984c393ff259-kube-api-access-j9597\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:32.603711 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.603691    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="381624bd-7373-48b0-9fb0-9b6b8d792790" path="/var/lib/kubelet/pods/381624bd-7373-48b0-9fb0-9b6b8d792790/volumes"
Jul 28 19:57:32.604067 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.604054    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5911e907-05c1-45d8-aefc-984c393ff259" path="/var/lib/kubelet/pods/5911e907-05c1-45d8-aefc-984c393ff259/volumes"
Jul 28 19:57:32.604364 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.604352    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddb26822-8d56-4554-b097-efbc3e9a99bd" path="/var/lib/kubelet/pods/ddb26822-8d56-4554-b097-efbc3e9a99bd/volumes"
Jul 28 19:57:32.719384 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.719366    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:57:32.724946 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.724927    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:57:32.725269 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.725246    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:57:32.730718 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:32.730704    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wcdpw_bde3f901-3495-4178-bedc-62e92c8aaecc/ovn-acl-logging/0.log"
Jul 28 19:57:39.560647 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.560619    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d66111-dhk42"]
Jul 28 19:57:39.561041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.560964    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ee31921-6bb1-4fe0-a13f-ee5f768f038b" containerName="payload-processing"
Jul 28 19:57:39.561041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.560975    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ee31921-6bb1-4fe0-a13f-ee5f768f038b" containerName="payload-processing"
Jul 28 19:57:39.561041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.560988    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0" containerName="maas-api"
Jul 28 19:57:39.561041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.560993    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0" containerName="maas-api"
Jul 28 19:57:39.561041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561001    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5911e907-05c1-45d8-aefc-984c393ff259" containerName="payload-pre-processing"
Jul 28 19:57:39.561041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561006    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5911e907-05c1-45d8-aefc-984c393ff259" containerName="payload-pre-processing"
Jul 28 19:57:39.561041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561013    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="381624bd-7373-48b0-9fb0-9b6b8d792790" containerName="maas-api"
Jul 28 19:57:39.561041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561017    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="381624bd-7373-48b0-9fb0-9b6b8d792790" containerName="maas-api"
Jul 28 19:57:39.561041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561024    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93579aec-c805-4eda-afdd-77830b1f5fa0" containerName="payload-pre-processing"
Jul 28 19:57:39.561041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561029    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="93579aec-c805-4eda-afdd-77830b1f5fa0" containerName="payload-pre-processing"
Jul 28 19:57:39.561041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561036    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a46bb93-f355-40b1-a7ca-1a4cbe34f974" containerName="payload-pre-processing"
Jul 28 19:57:39.561041 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561042    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a46bb93-f355-40b1-a7ca-1a4cbe34f974" containerName="payload-pre-processing"
Jul 28 19:57:39.561441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561051    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5025cdb-ab0d-457c-a948-11263ee3c692" containerName="payload-processing"
Jul 28 19:57:39.561441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561056    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5025cdb-ab0d-457c-a948-11263ee3c692" containerName="payload-processing"
Jul 28 19:57:39.561441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561063    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dee60a2-ae6f-4126-acae-05b0ead764c0" containerName="maas-api"
Jul 28 19:57:39.561441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561069    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dee60a2-ae6f-4126-acae-05b0ead764c0" containerName="maas-api"
Jul 28 19:57:39.561441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561077    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddb26822-8d56-4554-b097-efbc3e9a99bd" containerName="payload-processing"
Jul 28 19:57:39.561441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561082    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddb26822-8d56-4554-b097-efbc3e9a99bd" containerName="payload-processing"
Jul 28 19:57:39.561441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561131    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ee31921-6bb1-4fe0-a13f-ee5f768f038b" containerName="payload-processing"
Jul 28 19:57:39.561441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561138    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="381624bd-7373-48b0-9fb0-9b6b8d792790" containerName="maas-api"
Jul 28 19:57:39.561441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561144    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5025cdb-ab0d-457c-a948-11263ee3c692" containerName="payload-processing"
Jul 28 19:57:39.561441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561149    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5911e907-05c1-45d8-aefc-984c393ff259" containerName="payload-pre-processing"
Jul 28 19:57:39.561441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561153    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dee60a2-ae6f-4126-acae-05b0ead764c0" containerName="maas-api"
Jul 28 19:57:39.561441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561158    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="93579aec-c805-4eda-afdd-77830b1f5fa0" containerName="payload-pre-processing"
Jul 28 19:57:39.561441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561163    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddb26822-8d56-4554-b097-efbc3e9a99bd" containerName="payload-processing"
Jul 28 19:57:39.561441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561168    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a46bb93-f355-40b1-a7ca-1a4cbe34f974" containerName="payload-pre-processing"
Jul 28 19:57:39.561441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.561172    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9bce86b-34b1-4bbf-83cb-da0f33d0b0a0" containerName="maas-api"
Jul 28 19:57:39.565946 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.565928    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d66111-dhk42"
Jul 28 19:57:39.568767 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.568745    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s76n2\""
Jul 28 19:57:39.571042 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.571013    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d66111-dhk42"]
Jul 28 19:57:39.678953 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.678924    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23aa820e-a1d8-4f32-bc67-948a43823662-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-d66111-dhk42\" (UID: \"23aa820e-a1d8-4f32-bc67-948a43823662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d66111-dhk42"
Jul 28 19:57:39.779892 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.779853    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23aa820e-a1d8-4f32-bc67-948a43823662-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-d66111-dhk42\" (UID: \"23aa820e-a1d8-4f32-bc67-948a43823662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d66111-dhk42"
Jul 28 19:57:39.780525 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.780471    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23aa820e-a1d8-4f32-bc67-948a43823662-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-d66111-dhk42\" (UID: \"23aa820e-a1d8-4f32-bc67-948a43823662\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d66111-dhk42"
Jul 28 19:57:39.878764 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:39.878690    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d66111-dhk42"
Jul 28 19:57:40.004450 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:40.004425    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d66111-dhk42"]
Jul 28 19:57:40.006363 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:57:40.006330    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23aa820e_a1d8_4f32_bc67_948a43823662.slice/crio-3b7c21c82c4073e320507bd8fbe5821a0e293ea53c213bcc09f6959aad653e44 WatchSource:0}: Error finding container 3b7c21c82c4073e320507bd8fbe5821a0e293ea53c213bcc09f6959aad653e44: Status 404 returned error can't find the container with id 3b7c21c82c4073e320507bd8fbe5821a0e293ea53c213bcc09f6959aad653e44
Jul 28 19:57:40.505462 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:40.505433    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d66111-dhk42" event={"ID":"23aa820e-a1d8-4f32-bc67-948a43823662","Type":"ContainerStarted","Data":"3b7c21c82c4073e320507bd8fbe5821a0e293ea53c213bcc09f6959aad653e44"}
Jul 28 19:57:43.520899 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:43.520865    2561 generic.go:356] "Generic (PLEG): container finished" podID="23aa820e-a1d8-4f32-bc67-948a43823662" containerID="bdacd9044b92029928fe6e1c636d02ddbd044a7cf0d1d5087f94d460bc37c96b" exitCode=0
Jul 28 19:57:43.521261 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:43.520952    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d66111-dhk42" event={"ID":"23aa820e-a1d8-4f32-bc67-948a43823662","Type":"ContainerDied","Data":"bdacd9044b92029928fe6e1c636d02ddbd044a7cf0d1d5087f94d460bc37c96b"}
Jul 28 19:57:44.654605 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:44.654583    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d66111-dhk42"
Jul 28 19:57:44.826754 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:44.826680    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23aa820e-a1d8-4f32-bc67-948a43823662-maas-api-service-ca\") pod \"23aa820e-a1d8-4f32-bc67-948a43823662\" (UID: \"23aa820e-a1d8-4f32-bc67-948a43823662\") "
Jul 28 19:57:44.827056 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:44.827029    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23aa820e-a1d8-4f32-bc67-948a43823662-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "23aa820e-a1d8-4f32-bc67-948a43823662" (UID: "23aa820e-a1d8-4f32-bc67-948a43823662"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:57:44.927749 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:44.927725    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23aa820e-a1d8-4f32-bc67-948a43823662-maas-api-service-ca\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:45.531620 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:45.531584    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d66111-dhk42" event={"ID":"23aa820e-a1d8-4f32-bc67-948a43823662","Type":"ContainerDied","Data":"3b7c21c82c4073e320507bd8fbe5821a0e293ea53c213bcc09f6959aad653e44"}
Jul 28 19:57:45.531620 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:45.531622    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d66111-dhk42"
Jul 28 19:57:45.531620 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:45.531630    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b7c21c82c4073e320507bd8fbe5821a0e293ea53c213bcc09f6959aad653e44"
Jul 28 19:57:47.284463 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.284427    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"]
Jul 28 19:57:47.284934 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.284667    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695" podUID="bcac02cc-aa07-4c95-bd7b-6be5076b0c55" containerName="maas-api" containerID="cri-o://d0148edd30ea89e507cef32c7dc728079fcb6a4659be1befe02a1281b13a976a" gracePeriod=30
Jul 28 19:57:47.345075 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.345044    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"]
Jul 28 19:57:47.345415 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.345240    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv" podUID="7923b780-8184-4612-b7ee-c480ab60cab3" containerName="payload-processing" containerID="cri-o://ba699cf8101b011eeaefeb5bb5c80869cd6a1b236c62538288723d3d6736610a" gracePeriod=30
Jul 28 19:57:47.544972 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.544944    2561 generic.go:356] "Generic (PLEG): container finished" podID="7923b780-8184-4612-b7ee-c480ab60cab3" containerID="ba699cf8101b011eeaefeb5bb5c80869cd6a1b236c62538288723d3d6736610a" exitCode=0
Jul 28 19:57:47.545118 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.545015    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv" event={"ID":"7923b780-8184-4612-b7ee-c480ab60cab3","Type":"ContainerDied","Data":"ba699cf8101b011eeaefeb5bb5c80869cd6a1b236c62538288723d3d6736610a"}
Jul 28 19:57:47.546872 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.546847    2561 generic.go:356] "Generic (PLEG): container finished" podID="bcac02cc-aa07-4c95-bd7b-6be5076b0c55" containerID="d0148edd30ea89e507cef32c7dc728079fcb6a4659be1befe02a1281b13a976a" exitCode=0
Jul 28 19:57:47.547000 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.546916    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695" event={"ID":"bcac02cc-aa07-4c95-bd7b-6be5076b0c55","Type":"ContainerDied","Data":"d0148edd30ea89e507cef32c7dc728079fcb6a4659be1befe02a1281b13a976a"}
Jul 28 19:57:47.590145 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.590121    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"
Jul 28 19:57:47.611867 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.611614    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"
Jul 28 19:57:47.622760 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.622733    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d66111-dhk42"]
Jul 28 19:57:47.625680 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.625661    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d66111-dhk42"]
Jul 28 19:57:47.749902 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.749876    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-maas-api-tls\") pod \"bcac02cc-aa07-4c95-bd7b-6be5076b0c55\" (UID: \"bcac02cc-aa07-4c95-bd7b-6be5076b0c55\") "
Jul 28 19:57:47.749991 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.749920    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7923b780-8184-4612-b7ee-c480ab60cab3-plugins-config-volume\") pod \"7923b780-8184-4612-b7ee-c480ab60cab3\" (UID: \"7923b780-8184-4612-b7ee-c480ab60cab3\") "
Jul 28 19:57:47.749991 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.749950    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fcnj\" (UniqueName: \"kubernetes.io/projected/7923b780-8184-4612-b7ee-c480ab60cab3-kube-api-access-9fcnj\") pod \"7923b780-8184-4612-b7ee-c480ab60cab3\" (UID: \"7923b780-8184-4612-b7ee-c480ab60cab3\") "
Jul 28 19:57:47.749991 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.749978    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kzhv\" (UniqueName: \"kubernetes.io/projected/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-kube-api-access-7kzhv\") pod \"bcac02cc-aa07-4c95-bd7b-6be5076b0c55\" (UID: \"bcac02cc-aa07-4c95-bd7b-6be5076b0c55\") "
Jul 28 19:57:47.750168 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.750029    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-openshift-service-ca\") pod \"bcac02cc-aa07-4c95-bd7b-6be5076b0c55\" (UID: \"bcac02cc-aa07-4c95-bd7b-6be5076b0c55\") "
Jul 28 19:57:47.750301 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.750280    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7923b780-8184-4612-b7ee-c480ab60cab3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7923b780-8184-4612-b7ee-c480ab60cab3" (UID: "7923b780-8184-4612-b7ee-c480ab60cab3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:57:47.750534 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.750475    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bcac02cc-aa07-4c95-bd7b-6be5076b0c55" (UID: "bcac02cc-aa07-4c95-bd7b-6be5076b0c55"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:57:47.752113 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.752088    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7923b780-8184-4612-b7ee-c480ab60cab3-kube-api-access-9fcnj" (OuterVolumeSpecName: "kube-api-access-9fcnj") pod "7923b780-8184-4612-b7ee-c480ab60cab3" (UID: "7923b780-8184-4612-b7ee-c480ab60cab3"). InnerVolumeSpecName "kube-api-access-9fcnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:47.752180 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.752111    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bcac02cc-aa07-4c95-bd7b-6be5076b0c55" (UID: "bcac02cc-aa07-4c95-bd7b-6be5076b0c55"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:57:47.752180 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.752121    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-kube-api-access-7kzhv" (OuterVolumeSpecName: "kube-api-access-7kzhv") pod "bcac02cc-aa07-4c95-bd7b-6be5076b0c55" (UID: "bcac02cc-aa07-4c95-bd7b-6be5076b0c55"). InnerVolumeSpecName "kube-api-access-7kzhv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:57:47.850838 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.850786    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-maas-api-tls\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:47.850838 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.850807    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7923b780-8184-4612-b7ee-c480ab60cab3-plugins-config-volume\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:47.850838 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.850816    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fcnj\" (UniqueName: \"kubernetes.io/projected/7923b780-8184-4612-b7ee-c480ab60cab3-kube-api-access-9fcnj\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:47.850838 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.850825    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kzhv\" (UniqueName: \"kubernetes.io/projected/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-kube-api-access-7kzhv\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:47.850838 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:47.850834    2561 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcac02cc-aa07-4c95-bd7b-6be5076b0c55-openshift-service-ca\") on node \"ip-10-0-137-70.ec2.internal\" DevicePath \"\""
Jul 28 19:57:48.553095 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:48.553026    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"
Jul 28 19:57:48.553095 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:48.553060    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695" event={"ID":"bcac02cc-aa07-4c95-bd7b-6be5076b0c55","Type":"ContainerDied","Data":"1ea13b84c0a9b603447fe47f6d726c3ccf5dcd8dde05e8868f826e0659e4d4df"}
Jul 28 19:57:48.553594 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:48.553121    2561 scope.go:117] "RemoveContainer" containerID="d0148edd30ea89e507cef32c7dc728079fcb6a4659be1befe02a1281b13a976a"
Jul 28 19:57:48.554404 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:48.554383    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv" event={"ID":"7923b780-8184-4612-b7ee-c480ab60cab3","Type":"ContainerDied","Data":"a8ca2f3caf5e92a74896687c8110fa3b34925d78a65e52c9fd568827986be5cb"}
Jul 28 19:57:48.554516 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:48.554427    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"
Jul 28 19:57:48.561401 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:48.561383    2561 scope.go:117] "RemoveContainer" containerID="ba699cf8101b011eeaefeb5bb5c80869cd6a1b236c62538288723d3d6736610a"
Jul 28 19:57:48.579120 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:48.579099    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"]
Jul 28 19:57:48.583953 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:48.583935    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-d66111-5dc44dc96c-xf695"]
Jul 28 19:57:48.594999 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:48.594978    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"]
Jul 28 19:57:48.603448 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:48.603426    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23aa820e-a1d8-4f32-bc67-948a43823662" path="/var/lib/kubelet/pods/23aa820e-a1d8-4f32-bc67-948a43823662/volumes"
Jul 28 19:57:48.603887 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:48.603869    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcac02cc-aa07-4c95-bd7b-6be5076b0c55" path="/var/lib/kubelet/pods/bcac02cc-aa07-4c95-bd7b-6be5076b0c55/volumes"
Jul 28 19:57:48.604158 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:48.604143    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d66111-b6dd9f7c8-r7stv"]
Jul 28 19:57:50.603899 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:50.603871    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7923b780-8184-4612-b7ee-c480ab60cab3" path="/var/lib/kubelet/pods/7923b780-8184-4612-b7ee-c480ab60cab3/volumes"
Jul 28 19:57:54.808344 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:57:54.808272    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8694dc4c86-q47rl_22468a84-6db3-48b0-911f-b2d0cfd2862f/authorino/0.log"
Jul 28 19:58:00.653978 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:00.653948    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr_edcbf2e3-b1a1-4c87-b93f-b0c048f04797/copy-manifests/0.log"
Jul 28 19:58:00.665000 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:00.664963    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr_edcbf2e3-b1a1-4c87-b93f-b0c048f04797/manager/0.log"
Jul 28 19:58:02.793847 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:02.793814    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8694dc4c86-q47rl_22468a84-6db3-48b0-911f-b2d0cfd2862f/authorino/0.log"
Jul 28 19:58:04.244909 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:04.244843    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b4694559d-thlwz_33b22490-f769-431e-b092-90624a1b08a9/kube-auth-proxy/0.log"
Jul 28 19:58:04.582324 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:04.582250    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-jdvzc_bec6473a-99eb-4911-9557-3ba753ed35eb/payload-processing/0.log"
Jul 28 19:58:04.689513 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:04.689477    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76b66f6655-gwxld_91043986-5ac8-4cd3-b699-f7029f419bfc/router/0.log"
Jul 28 19:58:05.021338 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:05.021303    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-dx9rt_322324a7-e80f-456e-b0da-5ddc45698b55/main/0.log"
Jul 28 19:58:05.454785 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:05.454750    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-45c2b_c71dd818-a008-468d-b733-e23a8c047005/main/0.log"
Jul 28 19:58:05.559548 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:05.559523    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-lqngh_bbb29a1e-f3f7-4608-ad9e-6e9cb02a377d/main/0.log"
Jul 28 19:58:11.129987 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.129954    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cgg7r/must-gather-j9nk5"]
Jul 28 19:58:11.130385 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.130292    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23aa820e-a1d8-4f32-bc67-948a43823662" containerName="revoke-keys"
Jul 28 19:58:11.130385 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.130302    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="23aa820e-a1d8-4f32-bc67-948a43823662" containerName="revoke-keys"
Jul 28 19:58:11.130385 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.130318    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7923b780-8184-4612-b7ee-c480ab60cab3" containerName="payload-processing"
Jul 28 19:58:11.130385 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.130323    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7923b780-8184-4612-b7ee-c480ab60cab3" containerName="payload-processing"
Jul 28 19:58:11.130385 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.130332    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcac02cc-aa07-4c95-bd7b-6be5076b0c55" containerName="maas-api"
Jul 28 19:58:11.130385 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.130338    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcac02cc-aa07-4c95-bd7b-6be5076b0c55" containerName="maas-api"
Jul 28 19:58:11.130647 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.130396    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7923b780-8184-4612-b7ee-c480ab60cab3" containerName="payload-processing"
Jul 28 19:58:11.130647 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.130407    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcac02cc-aa07-4c95-bd7b-6be5076b0c55" containerName="maas-api"
Jul 28 19:58:11.130647 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.130413    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="23aa820e-a1d8-4f32-bc67-948a43823662" containerName="revoke-keys"
Jul 28 19:58:11.138432 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.138409    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cgg7r/must-gather-j9nk5"
Jul 28 19:58:11.139512 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.139468    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cgg7r/must-gather-j9nk5"]
Jul 28 19:58:11.141082 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.141040    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cgg7r\"/\"openshift-service-ca.crt\""
Jul 28 19:58:11.141204 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.141102    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cgg7r\"/\"kube-root-ca.crt\""
Jul 28 19:58:11.141980 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.141962    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cgg7r\"/\"default-dockercfg-jlth6\""
Jul 28 19:58:11.240200 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.240173    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qv7g\" (UniqueName: \"kubernetes.io/projected/721fa3de-078c-4d67-b962-5377079b49ab-kube-api-access-6qv7g\") pod \"must-gather-j9nk5\" (UID: \"721fa3de-078c-4d67-b962-5377079b49ab\") " pod="openshift-must-gather-cgg7r/must-gather-j9nk5"
Jul 28 19:58:11.240354 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.240206    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/721fa3de-078c-4d67-b962-5377079b49ab-must-gather-output\") pod \"must-gather-j9nk5\" (UID: \"721fa3de-078c-4d67-b962-5377079b49ab\") " pod="openshift-must-gather-cgg7r/must-gather-j9nk5"
Jul 28 19:58:11.341102 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.341073    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qv7g\" (UniqueName: \"kubernetes.io/projected/721fa3de-078c-4d67-b962-5377079b49ab-kube-api-access-6qv7g\") pod \"must-gather-j9nk5\" (UID: \"721fa3de-078c-4d67-b962-5377079b49ab\") " pod="openshift-must-gather-cgg7r/must-gather-j9nk5"
Jul 28 19:58:11.341229 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.341109    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/721fa3de-078c-4d67-b962-5377079b49ab-must-gather-output\") pod \"must-gather-j9nk5\" (UID: \"721fa3de-078c-4d67-b962-5377079b49ab\") " pod="openshift-must-gather-cgg7r/must-gather-j9nk5"
Jul 28 19:58:11.341529 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.341513    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/721fa3de-078c-4d67-b962-5377079b49ab-must-gather-output\") pod \"must-gather-j9nk5\" (UID: \"721fa3de-078c-4d67-b962-5377079b49ab\") " pod="openshift-must-gather-cgg7r/must-gather-j9nk5"
Jul 28 19:58:11.349054 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.349036    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qv7g\" (UniqueName: \"kubernetes.io/projected/721fa3de-078c-4d67-b962-5377079b49ab-kube-api-access-6qv7g\") pod \"must-gather-j9nk5\" (UID: \"721fa3de-078c-4d67-b962-5377079b49ab\") " pod="openshift-must-gather-cgg7r/must-gather-j9nk5"
Jul 28 19:58:11.450031 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.450003    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cgg7r/must-gather-j9nk5"
Jul 28 19:58:11.577593 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.577570    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cgg7r/must-gather-j9nk5"]
Jul 28 19:58:11.580049 ip-10-0-137-70 kubenswrapper[2561]: W0728 19:58:11.580022    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod721fa3de_078c_4d67_b962_5377079b49ab.slice/crio-d441dde0e2017d3a808aff40c8af7536e8af0613060f3b74644c8e1feb709db4 WatchSource:0}: Error finding container d441dde0e2017d3a808aff40c8af7536e8af0613060f3b74644c8e1feb709db4: Status 404 returned error can't find the container with id d441dde0e2017d3a808aff40c8af7536e8af0613060f3b74644c8e1feb709db4
Jul 28 19:58:11.659439 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:11.659408    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cgg7r/must-gather-j9nk5" event={"ID":"721fa3de-078c-4d67-b962-5377079b49ab","Type":"ContainerStarted","Data":"d441dde0e2017d3a808aff40c8af7536e8af0613060f3b74644c8e1feb709db4"}
Jul 28 19:58:12.667684 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:12.667650    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cgg7r/must-gather-j9nk5" event={"ID":"721fa3de-078c-4d67-b962-5377079b49ab","Type":"ContainerStarted","Data":"8d7d7cb8699bf0e26c02ae94e0e52b68f97ebc89a473cc940431abffa69b386e"}
Jul 28 19:58:13.674622 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:13.674580    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cgg7r/must-gather-j9nk5" event={"ID":"721fa3de-078c-4d67-b962-5377079b49ab","Type":"ContainerStarted","Data":"b79ddddb4be19322534887ee85110f3b9a54ef8b5cfd5cca3a10603a78ff50f2"}
Jul 28 19:58:13.707291 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:13.707243    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cgg7r/must-gather-j9nk5" podStartSLOduration=1.774731875 podStartE2EDuration="2.707225082s" totalImagesPullingTime="932.493207ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:58:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:58:13.706747664 +0000 UTC m=+3341.653232532" watchObservedRunningTime="2026-07-28 19:58:13.707225082 +0000 UTC m=+3341.653709949"
Jul 28 19:58:14.106001 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:14.105967    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-68mlb_563e6899-0366-4322-a41e-b59d7a4a4b12/global-pull-secret-syncer/0.log"
Jul 28 19:58:14.295754 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:14.295717    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lxcmx_40534a0a-48dc-48a6-bfd0-3d0eb410a4c1/konnectivity-agent/0.log"
Jul 28 19:58:14.353879 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:14.353845    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-70.ec2.internal_683f92978e5f7131a5a5e68a817885c9/haproxy/0.log"
Jul 28 19:58:19.747506 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:19.746873    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8694dc4c86-q47rl_22468a84-6db3-48b0-911f-b2d0cfd2862f/authorino/0.log"
Jul 28 19:58:22.088064 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.088023    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-dr8wz_a4d213bf-895d-4904-9052-493dd676de9d/kube-state-metrics/0.log"
Jul 28 19:58:22.107779 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.107746    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-dr8wz_a4d213bf-895d-4904-9052-493dd676de9d/kube-rbac-proxy-main/0.log"
Jul 28 19:58:22.133624 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.133579    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-dr8wz_a4d213bf-895d-4904-9052-493dd676de9d/kube-rbac-proxy-self/0.log"
Jul 28 19:58:22.164001 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.163956    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7759958bbb-455qk_04a819c6-fd34-4174-ac7a-2459c9c96e71/metrics-server/0.log"
Jul 28 19:58:22.193269 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.193228    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-btjkv_13f9a1c2-3b3a-4e1a-aa99-f4dbec47256a/monitoring-plugin/0.log"
Jul 28 19:58:22.301914 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.301887    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l62l7_8b504f9a-cac5-44ec-9ce1-f17176709745/node-exporter/0.log"
Jul 28 19:58:22.321920 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.321881    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l62l7_8b504f9a-cac5-44ec-9ce1-f17176709745/kube-rbac-proxy/0.log"
Jul 28 19:58:22.346328 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.346241    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l62l7_8b504f9a-cac5-44ec-9ce1-f17176709745/init-textfile/0.log"
Jul 28 19:58:22.449184 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.449151    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-qzwpn_522f46e3-a690-493f-9b7f-50668ce33072/kube-rbac-proxy-main/0.log"
Jul 28 19:58:22.473006 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.472980    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-qzwpn_522f46e3-a690-493f-9b7f-50668ce33072/kube-rbac-proxy-self/0.log"
Jul 28 19:58:22.494047 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.494011    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-qzwpn_522f46e3-a690-493f-9b7f-50668ce33072/openshift-state-metrics/0.log"
Jul 28 19:58:22.541363 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.541326    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f68efc2c-2a16-4f1e-9e8a-27890cff3931/prometheus/0.log"
Jul 28 19:58:22.566924 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.566887    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f68efc2c-2a16-4f1e-9e8a-27890cff3931/config-reloader/0.log"
Jul 28 19:58:22.591223 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.591198    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f68efc2c-2a16-4f1e-9e8a-27890cff3931/thanos-sidecar/0.log"
Jul 28 19:58:22.621566 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.620547    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f68efc2c-2a16-4f1e-9e8a-27890cff3931/kube-rbac-proxy-web/0.log"
Jul 28 19:58:22.642621 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.642597    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f68efc2c-2a16-4f1e-9e8a-27890cff3931/kube-rbac-proxy/0.log"
Jul 28 19:58:22.664673 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.664637    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f68efc2c-2a16-4f1e-9e8a-27890cff3931/kube-rbac-proxy-thanos/0.log"
Jul 28 19:58:22.687794 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:22.687764    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f68efc2c-2a16-4f1e-9e8a-27890cff3931/init-config-reloader/0.log"
Jul 28 19:58:24.109627 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.109602    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-4vr96_56684944-6fb3-4f87-91b7-b3f25ec77d7a/networking-console-plugin/0.log"
Jul 28 19:58:24.705243 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.705210    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"]
Jul 28 19:58:24.708685 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.708654    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/2.log"
Jul 28 19:58:24.718755 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.718726    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:24.721929 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.721884    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-xvkq8_23ad8ae2-6a98-4bda-988d-ac67c7be5946/console-operator/3.log"
Jul 28 19:58:24.723965 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.723942    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"]
Jul 28 19:58:24.805652 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.805606    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b697d586-3123-461c-82c5-c275cbea0d3e-sys\") pod \"perf-node-gather-daemonset-g2zpn\" (UID: \"b697d586-3123-461c-82c5-c275cbea0d3e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:24.806187 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.806161    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b697d586-3123-461c-82c5-c275cbea0d3e-lib-modules\") pod \"perf-node-gather-daemonset-g2zpn\" (UID: \"b697d586-3123-461c-82c5-c275cbea0d3e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:24.806387 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.806362    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b697d586-3123-461c-82c5-c275cbea0d3e-podres\") pod \"perf-node-gather-daemonset-g2zpn\" (UID: \"b697d586-3123-461c-82c5-c275cbea0d3e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:24.806566 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.806531    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b697d586-3123-461c-82c5-c275cbea0d3e-proc\") pod \"perf-node-gather-daemonset-g2zpn\" (UID: \"b697d586-3123-461c-82c5-c275cbea0d3e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:24.806739 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.806720    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvffp\" (UniqueName: \"kubernetes.io/projected/b697d586-3123-461c-82c5-c275cbea0d3e-kube-api-access-zvffp\") pod \"perf-node-gather-daemonset-g2zpn\" (UID: \"b697d586-3123-461c-82c5-c275cbea0d3e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:24.908062 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.908027    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b697d586-3123-461c-82c5-c275cbea0d3e-sys\") pod \"perf-node-gather-daemonset-g2zpn\" (UID: \"b697d586-3123-461c-82c5-c275cbea0d3e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:24.908062 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.908072    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b697d586-3123-461c-82c5-c275cbea0d3e-lib-modules\") pod \"perf-node-gather-daemonset-g2zpn\" (UID: \"b697d586-3123-461c-82c5-c275cbea0d3e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:24.908268 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.908099    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b697d586-3123-461c-82c5-c275cbea0d3e-podres\") pod \"perf-node-gather-daemonset-g2zpn\" (UID: \"b697d586-3123-461c-82c5-c275cbea0d3e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:24.908268 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.908194    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b697d586-3123-461c-82c5-c275cbea0d3e-sys\") pod \"perf-node-gather-daemonset-g2zpn\" (UID: \"b697d586-3123-461c-82c5-c275cbea0d3e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:24.908268 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.908234    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b697d586-3123-461c-82c5-c275cbea0d3e-proc\") pod \"perf-node-gather-daemonset-g2zpn\" (UID: \"b697d586-3123-461c-82c5-c275cbea0d3e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:24.908430 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.908274    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b697d586-3123-461c-82c5-c275cbea0d3e-proc\") pod \"perf-node-gather-daemonset-g2zpn\" (UID: \"b697d586-3123-461c-82c5-c275cbea0d3e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:24.908430 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.908278    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvffp\" (UniqueName: \"kubernetes.io/projected/b697d586-3123-461c-82c5-c275cbea0d3e-kube-api-access-zvffp\") pod \"perf-node-gather-daemonset-g2zpn\" (UID: \"b697d586-3123-461c-82c5-c275cbea0d3e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:24.908430 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.908236    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b697d586-3123-461c-82c5-c275cbea0d3e-podres\") pod \"perf-node-gather-daemonset-g2zpn\" (UID: \"b697d586-3123-461c-82c5-c275cbea0d3e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:24.908430 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.908236    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b697d586-3123-461c-82c5-c275cbea0d3e-lib-modules\") pod \"perf-node-gather-daemonset-g2zpn\" (UID: \"b697d586-3123-461c-82c5-c275cbea0d3e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:24.915644 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:24.915622    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvffp\" (UniqueName: \"kubernetes.io/projected/b697d586-3123-461c-82c5-c275cbea0d3e-kube-api-access-zvffp\") pod \"perf-node-gather-daemonset-g2zpn\" (UID: \"b697d586-3123-461c-82c5-c275cbea0d3e\") " pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:25.035719 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:25.035635    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:25.212949 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:25.212923    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"]
Jul 28 19:58:25.768180 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:25.768130    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn" event={"ID":"b697d586-3123-461c-82c5-c275cbea0d3e","Type":"ContainerStarted","Data":"0fa3e2a080fe13bee4f6b0e48a56b5f7729432585b2ba7d4fcc9533d0be7470e"}
Jul 28 19:58:25.768383 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:25.768205    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn" event={"ID":"b697d586-3123-461c-82c5-c275cbea0d3e","Type":"ContainerStarted","Data":"ec62d6bcf7cba8555bed628a08d15d4fe809ae1d1ba6493790343fd2130963a1"}
Jul 28 19:58:25.768383 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:25.768294    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:25.784649 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:25.784594    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn" podStartSLOduration=1.784578102 podStartE2EDuration="1.784578102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:58:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:58:25.783098813 +0000 UTC m=+3353.729583707" watchObservedRunningTime="2026-07-28 19:58:25.784578102 +0000 UTC m=+3353.731062969"
Jul 28 19:58:26.853047 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:26.853014    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-glvld_1352efd5-8baf-4196-8e4b-4c1b379eca9f/dns/0.log"
Jul 28 19:58:26.874223 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:26.874200    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-glvld_1352efd5-8baf-4196-8e4b-4c1b379eca9f/kube-rbac-proxy/0.log"
Jul 28 19:58:26.986751 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:26.986723    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j5pqn_1bc180f1-52ba-4d7f-a886-025e5a39af95/dns-node-resolver/0.log"
Jul 28 19:58:27.524942 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:27.524916    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7468fc8f9d-zxtzw_0335142c-a57c-4d34-bea7-8c022dba103c/registry/0.log"
Jul 28 19:58:27.595615 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:27.595583    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ffk9d_e45507b7-c87c-44b9-b495-b5d812d57324/node-ca/0.log"
Jul 28 19:58:28.832918 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:28.832888    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b4694559d-thlwz_33b22490-f769-431e-b092-90624a1b08a9/kube-auth-proxy/0.log"
Jul 28 19:58:28.957502 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:28.957461    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-jdvzc_bec6473a-99eb-4911-9557-3ba753ed35eb/payload-processing/0.log"
Jul 28 19:58:28.980266 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:28.980239    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76b66f6655-gwxld_91043986-5ac8-4cd3-b699-f7029f419bfc/router/0.log"
Jul 28 19:58:29.793830 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:29.793792    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c2fr5_c619cfcb-63dc-4707-beeb-ceeb823e29a9/serve-healthcheck-canary/0.log"
Jul 28 19:58:30.357831 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:30.357799    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tg8vp_e7bf5170-55ae-4c7d-860e-1275f3f63dce/kube-rbac-proxy/0.log"
Jul 28 19:58:30.380542 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:30.380471    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tg8vp_e7bf5170-55ae-4c7d-860e-1275f3f63dce/exporter/0.log"
Jul 28 19:58:30.400494 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:30.400455    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tg8vp_e7bf5170-55ae-4c7d-860e-1275f3f63dce/extractor/0.log"
Jul 28 19:58:31.788351 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:31.788321    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cgg7r/perf-node-gather-daemonset-g2zpn"
Jul 28 19:58:33.186202 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:33.186178    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr_edcbf2e3-b1a1-4c87-b93f-b0c048f04797/manager/0.log"
Jul 28 19:58:33.204163 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:33.204084    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b9c87dd8f-p2nfr_edcbf2e3-b1a1-4c87-b93f-b0c048f04797/copy-manifests/0.log"
Jul 28 19:58:34.621893 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:34.621859    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7fbf845658-x9wsg_e07ee69b-85e7-485f-8072-ea7010acea97/manager/0.log"
Jul 28 19:58:34.886313 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:34.886282    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b4694559d-thlwz_33b22490-f769-431e-b092-90624a1b08a9/kube-auth-proxy/0.log"
Jul 28 19:58:34.979133 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:34.979098    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-jdvzc_bec6473a-99eb-4911-9557-3ba753ed35eb/payload-processing/0.log"
Jul 28 19:58:34.993646 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:34.993622    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76b66f6655-gwxld_91043986-5ac8-4cd3-b699-f7029f419bfc/router/0.log"
Jul 28 19:58:36.692257 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:36.692224    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b4694559d-thlwz_33b22490-f769-431e-b092-90624a1b08a9/kube-auth-proxy/0.log"
Jul 28 19:58:36.792558 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:36.792527    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-jdvzc_bec6473a-99eb-4911-9557-3ba753ed35eb/payload-processing/0.log"
Jul 28 19:58:36.814441 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:36.814416    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76b66f6655-gwxld_91043986-5ac8-4cd3-b699-f7029f419bfc/router/0.log"
Jul 28 19:58:40.905066 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:40.905037    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25rm8_83181e12-1e53-4972-8b1b-6ab591b30cf7/kube-multus-additional-cni-plugins/0.log"
Jul 28 19:58:40.925280 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:40.925250    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25rm8_83181e12-1e53-4972-8b1b-6ab591b30cf7/egress-router-binary-copy/0.log"
Jul 28 19:58:40.944696 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:40.944677    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25rm8_83181e12-1e53-4972-8b1b-6ab591b30cf7/cni-plugins/0.log"
Jul 28 19:58:40.963869 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:40.963850    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25rm8_83181e12-1e53-4972-8b1b-6ab591b30cf7/bond-cni-plugin/0.log"
Jul 28 19:58:40.983474 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:40.983454    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25rm8_83181e12-1e53-4972-8b1b-6ab591b30cf7/routeoverride-cni/0.log"
Jul 28 19:58:41.003362 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:41.003346    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25rm8_83181e12-1e53-4972-8b1b-6ab591b30cf7/whereabouts-cni-bincopy/0.log"
Jul 28 19:58:41.021970 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:41.021943    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25rm8_83181e12-1e53-4972-8b1b-6ab591b30cf7/whereabouts-cni/0.log"
Jul 28 19:58:41.471812 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:41.471787    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rbq99_3c12d37f-cd16-4f11-9681-eaab31e63548/kube-multus/0.log"
Jul 28 19:58:41.585496 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:41.585438    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hntnt_0b4c98d2-1b87-4f67-8881-b4e5639b9895/network-metrics-daemon/0.log"
Jul 28 19:58:41.603417 ip-10-0-137-70 kubenswrapper[2561]: I0728 19:58:41.603394    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hntnt_0b4c98d2-1b87-4f67-8881-b4e5639b9895/kube-rbac-proxy/0.log"